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" ^SUMMARY 



.A Study was conducted to investigate ^d statisticially 
•validate a performance measuring systeja in uhe Tactical Air 
^Command Air Combalt Engagement Simu^^ar^r I air combat ma-aeu- 
vering (ACM) training program at Vought Corporation, Dallas, 
Texas. The study utilized a 12 week sample oV'89 student ^ 
pilots in an experiment tc statistically validate an objec- 
tive performance measure of air combat skill, compare the.: 
c^jective measure to the subjective judgement of ACM skill 
made by instructor pilots, to investigate^mprovements as a 
measure of ACM skill, and to evaluate its utility as^V>train 
ing aid. 

Statistifial inethodologies of ridge regression and disy 
criminant analyses " were employed "to assess the quantitative 
and- qualitative characteristics of the measure of ACM skill 
in the simulator. ' * ^" • ' i ^ 

Bc^.ckground " ' 

A scoring systejna tepcied the jGood Stick Indbex (GSI) is. 
.Used as an indicator of pilot air combat skiJLl in the TAC 
ACES I simulator training prograin.' The GSI was developed 
jointly "by the Tactical Air Command and the Vought Corpora- 
ti^on utilizing four subjecriveJLy' chos^en and equally weighted 
parameters which to the experienced pil^ot are intuitive in- 
dicators of air combat' skill. Thjp,four parameters are 1) 
time in gun firing - envelope, 2> average mil error, 3) offe'n-* 
sive/defensive time, an<^ 4) tfme to' first kill - objective 
measures obtained Huring' student pil.ot scoring sessions 
against programmed tcirget maneuvers . The TAC ACES * I 



training pr rair ii 
irvent tourna.-''ii<=3t me: 
ano^er. Tr.. c-^ke 
tion .event ^^Icrs 
ted) resulti. -n ^ 
The GS: :cre, 
placement ^ -^rarec 
random frequent r^. 



on ^ . 0=^:. -^y*^ one-on-cie 'f ree enga:;?= 
r e *icuce:nt pilot is matched aga.^^ 
2i .c- ||,c jrnairient *is a dcub^e elimi.'iE 
-1*5 twc engagements ro'be eiir^- 
^ - Xi- ^. rjie r . ^ , ^ 

;':'._=L r= = predictor c: t,urkey shco 
ic. :ihe winner at greater th;ar 



.on 



UL^y was^ cpn^ucted to statisti la 
^ \ \ ■ ■ t 

pr=z'^ crior of •'turkey shoct winner, i 



vestigate imprr-v^e 
of each of th^ fouv 



Specifics . • * ' . 

The GSI -^iai 
validate the G31 as 

1.7. : . GSI by varj^ing the weighting 
^ame- :rs , and introducing^ additiona 
parauneters as candid ates f ^ an ^.mproved predictor of t^r 
shoot wiijner- The ali^dattr : GSI was compared with the t 
key shoot sudsnt 7 ^aceiuer ' pred^ction^j>pf the dnstructci 
pilot to assess it', igreei^en-:: wifeii expert opinion^- 

In order %> ler evaluate the^potential uti'lity cr 
the^GSI; fou:-*^ -roup^ngs of rurkey shpot pl^bements- in et 
class o£ eic^::f^ stu:u^ts were investigated; 

1 nn^ rs^ ' / - . - ' 

• 2 •-nne rs and Runners-Up (Finalists) 
3 >pe>-Half ^^^i-Finalists) 
4) drr^le Rankings. 
Data used _r tne study were collected ^.during • the 12 
class (12 weej^) ..ample ^rom 3 April 1978 through 23 June 
,1978^ These z.az.c ^er^ objective measu^tes of performanfce, 
in the simifla-or, i-emograohic (background) data obtained 
by student questicr^aaire / and instructor pilots' predic- 
tions of turkey shocr . placemei^t of students within- each 
class. The obi^Cwive* measures were obtained from scoring 
sessions on Mondays, immediately after 'brief ing and hands- 



rainiliarizatior. . and ir Fridays .^t prior tc -he turkey 
«hoot exercise. In -u:: of, the - classes/ an ad-i- 
-ional scoririg sess vas 'helc Wednesdays lo better 

assess learnin trei ^i;* iil-^he s ^-t ator. - 

The TAG AC^^S I . aining sy ar Was cons^ster^t 

^-iroughout the ^Kpe z^ent as atr:=st-- to by^ th^ Ch-^ef In> 
i true tor* Pilot: .-s-ructor pi , :.-z — ovided individual 

iristruction tc s^udent^ ^ cor. r/ating in areas of 

r3cognized'de « -z:l^s. The s-uc r-.s' were aware of the 
s'&orin^ sessi a- , at were unaware of the intended use of ^ • • 
the acquired -."~a. . - " 

Results 

* The firs ^a- is^iiqal analy? ? performed ^determined 
:h^ p;»edi^M^icr: apa^lity df^^e ^ually v/eighted, four-- . 
paramktM*- GSI cor:^ obtained in r-^-^-day scoring sessions. 
The results w^a compared^ to the s -i^jective' student turkey 
shoot rank pr=^crions ,of the ins ::ructor pilots. The analy- 
sis showed th^ GSI score, using I riday only data, to j^riedict 
the turkey ^^oot winner " with a ^5 percent probability (one 
in four). There was no statistical difference between the 
GSI and. the instructor pilot -prediction capabilities, 

A second analy&is summed the GSI score obtained on 
Friday to the GSI score^ obtained on Monday 'and* optimally * • 
weigfTted the combined score. A significarit increase- in 
^ probability of correct turkey shoot placement was observed 
at. about 66 percent (two in three) . " • ' 

\^third analysis used the' four individual pa- amet^s 
o^ each G^I. score for Monday and Friday (a toWl^ of eight . 
terms) and optimally-. weighted each individual parameter. 
The' results increased the prec^iction of turkey- shoot place-, 
meat to about* 75 percent (three ia fp^r) , the b^st predictiqn 



which could be obtained with the .four^yparameters- fi;^^^- 

tiyely.- c^aosen as iri:flicators of ACM skill. > ' 

In the fourth statistical analysis;, a s^t of 4^ . ec^' 

•-tive .measi±res taken during. ^each scoring session wete ^ - 

'troduced>to the discriminant model .as potential pr-ed^_r-or 

candidates. Included- in the data set were the four rr.ra- 

meterg in the original and improved GSI score, .The a. al^ i^is 

derived an optimal* predictor with about 8C percent pr ,j:ral::L- ; 

lity of correct turkey shoot placement/ Further, a ^^-r cz 

12 subjectively "chosen demographic (background) dara tamed 

. from student ques'tionnaires introduced as potently coiitr^bu- 

♦/ 1;or Candida tes^jLi^ -the expanded list of candidates. ii^- _ro- 

^ . bability of i:orrect turkey ^shoot placement remained abcu:: 

SO^percent however, background parameters of^ tot i :i.me 

in fighter aircraft, time in the F-4 aircraft, And me 'om- 

ber of sorties flowrr'in the last thirty days, replaced mree 

of the termg in the optimal ob-Jjective predictor sec re. This 

result, reinforces thd predictor model as a measure jf ;^^:-ct 

• ACM skill. ^ • . ' - . ^ ^ . 

» 

The statistically 'validated GSI was used^n the- fxr -1 
analysis to obt&in a measure of learning trends in the 
/lator. • A third scoring session on Wednesday, in /additicn to 
the Monday and Friday data, enabled an evaluation o± ski-l- 
develppmen't in the simulator over^ the week's training period. 
A quadratic* fit through the means of individual scores ob— 
.'^t^ned on -the three dlays Showed'* defitiite positive g2;oup 
learning (edumetric trend) . The d2.s tribution of individual 



scores wa& sedn to" converge, or* group Scloser together, - 

»from Moijday "to Friday.. ' The-, sjJLope* zt the quadratic fit ' 

(i approached zero pn.E'mday, which in^icaftes tliat one week's 

• training* m the simulator was, optimal fo'r the classes sub- 

\ • • : ' *^ ^ ' , V ^ p. 

gected to the investigation. , \ . 



Conclusions 



The overall ana.l:~e£: the study showed the GSI to be 
a measur^ of AGM skili:^wi-sh contributing parameters consis- f 
tent with intuitive ex^per-z ppiinion-and wfth an acceptable 
level of accurate assessii(ent' of skill in theVsimulator. The 
GSr .score ' is shown "to be useful in, evaluating individual 
\and group learning wit..iin training 5)rograins in ACM, and the 
individual parameters comprising the GSI score can be used 
as teaching guides. ^ . ^ 

A recommendatior is made t6^ utilize the algorithmp an<i 
similar techniques methodologies as presented in this 

study to derive perf cnwance^easurement systems, foj^ the 
Simulator for Air-to-Air Combat at j^uke AFB ar^^the Air 
CoAibat Maneuvering .Instrum^tatioh (ACMI) flapg^at Nellis^AFB. 
When an objective performance measure can be obtained^ for 

ACM'>in ; the air, then^^n objective measure, of trahsfer of 

. • ■ . • *" - ^ • . 

training between the >simulatbr--an(i the aircraft can be' as"- 

^ . / y\ «^ 

certained. 

.Applications of the te^ch^iiquBs of the study can aiso^be 
applied t6 other ACM simulators andN^ther, ty^s/*of flight 
simulators^ to aqhieve like meaisures. of ski^l .in a variety 
of 4f lying tasks. 

- . ', • • - : " ■ ■ \ \ 



. .'This report documents the ta^ks performed under con- , 
tract. F34601-77-A-0l76-KW01^, the Good Stick Index Validation. 
Study. The Vought Corporation, Dallas, Texas, has been 
und6r contract with the USAF Tactical Air Command /(ta^;) "^o 
furnish the Air Combat Engagement Simulator (ACES) facility 
^ in support of TAC' air^ombat. training during the data collec- 
tion phase of this study, A pilot 'performance scoring sys- 
tem^ "the Good Stick I^dex (GSI) , was developed earlier ;for 
the purpose of ^ predicting relative pe'rfj^aqmance of - student . 
pilots in a free engagement competition within ea<5h class of 
eight pilots. Initially, foyr parameters of pilot ^perfor- , 
mance were used to compute a GSI score for each/ pilot. ^ 
These parameters were selected subjectively and were em- , 
pirically weighted in:;the scoring equation.. " There had been 
no previous effort to statistij::ally validate the predictive 
ability of the GSI eqXiation. " * 

The contractor wishes to acknowledge the technical 
guidance ajld assistande- provided by Mr. ^ Robert -E. Coward/ 
Contract .Manager and*Co-Author , Fly'ing Trainl^rig pi vision , of 
.the Ai-r Force Human Resources ;rrabor^tory, cmd^ thfe program 
training, planning, and* scheduling dnilerf ace ^ of TAC ACES I 
-personnel provided by Lt. Col. John K. Sloan JI of tl>e Air 
^^rcfe Tactical Fighter Weapons'^ Cehter. • 
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I. .INTRODUCTION 

■ ■» 

The^Good Stick Index (GSI) is a Numerical iridex^ devel--* 

Jy ( 

oped to measure student pilot prof iciencies in -sirrmiated . 
one-on-one air combat. The GSI, as oriainally formulated tfy 
the^ Vought Corporation, Dallas, Texas , '^l^sists " a£' four \ 
objective performance parameters measured (^uring USAF Tacti- 
cal Air Xlommand (TAG) Air Combat Engagement Simulatpf (ACES) 

■r 

I training, , 

The fpur parameters comprising the GSI were subjec- 
tively chosen and, from data 'obtained over many classes, 

. . / • * ■ ■ ^ • 

empirically related, to derive a priedictor of the "winner" 

or "runner-up" in the double elimination one-on-one f ree ^ 
engagement tournament held at the conclusion of each train- 
ing session. This derived relationship ap^gars to predict 
the winner or runner-up of the double elimination free engage 
meht " turjcey-shoot" with greater than ofandom frequieno^ 

This study investiga;tes the pjredictive . ability of the 
empirically derived relationship as* a predictor ^"f turkej^'^ 
shoot winner by utilizing statistical analysis meth(^_ds, 
Furt^r, the study derives, through statistical tec^jiiques, 
the optimal predictor indices using the, original four sub- 
jectively chosen parameters and then derives optimal pre- 
dictors from ^lan expanded set of objective measures, which 
include the four parameters originally chosen, , 

These* analyses were performed using data collected- from 
12 classes of students in an experiment representative of 
TAC ACES I training. Input data , fidelity was assured by 
(a), certification that there was adherence to the training 
syllabus by the Instructor Pilots (IPs), (b) certification 
that there were no hardware anomalies, and (c) certifica- , 
tipn^ that there were no software anomalies unaccour|^ted for 
during the control period. 



. Additional analyses were performed to obtain correla- 
tion^ of student pilot background data and iX^^sub jective 
predictions of studeijit ranking relative to GSI s&ores and 
actual turkey shoot, rankings. 

Four of the 12 classes in the experiment were struc- 
tured to collect/atdditional edilnetric and psychome'trie ^ ' 
parameters in order to obtain a greater measure of i^|divi- 
dual ari4 group trahsfer of training in the simulator. 

The optimal GSI predictors, as derived by statistical 
analyses of the experiment data, are evaluated as a ^redic-^ 

tor. Using previous class sessions as a data base to "a 

* ' ,. ■ 

limited degree, an assessment is made of actual turkey shoot 
prediction capability. \ ^ . » 

BACKGROUND 

^The TAG ACE^ I , training program is conducted by the 
Tactical Air Command using the Vought CorporaLtion fixiBd 
base air combat simulator (Figure 1). The program utilizes 
two F-4 configured cockpits with full instruments and wea- 
pon systems indicators necessary for air-tb-air combat 
simulation in a functional mode. The software modeling 
is for F-4D and F-4E aircraft flight characteristic. In 
addition, a MIG 21 isJ^podeled to provide training in dis- 
similar.^ aircraft engagements. 

Facility Description 

T\ie Vought Air Combat Simulator, Figure 1,^ c^sistsi of 
two cockpits, each situated within 16-f oot-diameter sph^ri- . 
bal screens. Overhead projectors provide dynamic earth/sky 
horizon scenes and an* image of the opponent's aircraft. 
The aircraft target is a high- resolution color image pro- 
vided by the Opaque Target Optical Project System (OTOPS) , 

^ i5 „ 



Figure 1. The Voug:" - Azr Combat Simulator 
recently developed* oy Voumt. Each pi»Lot wears a %-suit 
and -sits -cn a g-sea-. ^As a zilc z increases the load factor 
on hi3 aircraft, hi; g-su. u _nfiites and his g-seat de- 
flates,^ The visual cxspl^y dims as 'a function of g and 
time and finally blacxs out, t^fi^arg^^ imagb the last 

.to go The g-seat also provides a 'buffet cue, beginning 
as a r gh f^quency nibble, ir creating in amplitude and 
decreasing in frequency ae peretration in^o the buffet area 
occurs. Ea,c cockpit is equipped wi^h fire control switch- 
ology which eflects the F-4E , number^ 556 and subsequent,- 
as mo_ified T.o; 1F-4E-556. . ~ ' . 
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On-linq firing and "hit cues, engine, aircraft, and we - 
pon sounds add to the realism of ^ the simulated aiz combat^^ar 
a separate b.ullet . model includes the t;Lme^of flic; t. Wea- 
pon ^re^^L^syn extends to the heit and rada$ missiles, too, 
as a miss will be Scored if the aircraft target exceeds the 
missile turning/tracking capabilities befoVe^ the time of 
flight has elapsed. A ' pi^lot scoriAg system called the GSI 
measures the Relative air combat skills of the pilot. 

A unique Ins tructor Pilot (IP) station that is mobile 
^d that can be operated from alongside the cockpit pro- 
vides the IP a matchless vantage point. The IP station, 
provides complete control of the simulation, includ^Lng 
operate, freeze oa< reset , 'replay , data recording, video 
recording, and options to record and play baqk preprogrammed 
or canned target trajectories . It al-Q contains the engage- 
ment scene wjiich can be recorded cn video ^cassettes , • along 
with the audio from both cockpit^ and the IP, for subsequent 
replay and ^debrief ing. \ ^ '^>" 

Training Sessions 

Typically, the TAG ACES I -raining session^ is sche- 
aled for one week and consists of eight student pilots and 
-nree IP^s. Each student accumulates a minimum of ten hours 
-f claissroom and hancs-on train^-ng in air-to-air combat. 
Tr^o student pilots train simialtaneously in the dual dome 
zwo-cockpit facility. Each student pilot is normally in- 
structed by an individual IP, but a single IP can instruct 
"^both pilots simultaneously./ Training data ar.e normally 
recorded while "flying" against a target with prep/ogrammed 
flightpaths. A kill is "scored" by^^iins, heat missile, 
r^dar missile^ or^ ground strike; , 

. / ' ^ 17 ■ . . - 
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The atudent pilot undergoes initial briefings And- s^-' 
ulator fai^iliarity sessions on the first day of the five 
d^s of training. After .becoming fa^H^r with the" simu.a- 
tor characteristics through tal^ haft ds -on session/ the stu- 
dent ±s "scored" against a serieg of canned target maneuver's 

The studerjfs -initial performance is recorded by computer 
and storea on magnetic tape. ■ ... 

The tf^aining progresses during the^we^' iri accord with 
the TAG'ACE^ I tfaining syllabi^.. Theytinal day of train- 
ing/ S:h6 fifth day, consists ofSa econd scoring session 
with ekch'stud^t pilot competing .gainst canned target!' 
,m^neuvers as was initially doAe or. the first day of train- 
ing. The class training culminates by a double eliminatio 
competition; dr turkey shoot, where each student compete 
against the others in one-on-one fr^e ehgage'ments until 
eliminated or a\winner is decided.* 

Background dat:a ar6 •collected on each pilot undergo in 
TAG ACES I training. In addition, each student pilot is * 
asked to^ sul^jectively evaluate the simulator perf-onriances 
in ComparisQn to the actual aircraft. Subjective evalua- ' 
tions of 'the training effectiveness and pc-ential impro\-e- 
ments are also, solicited. These data are recorded on ap- 
propriate questionnaire forpis and, tr^nsmit-ad to TAG, an 
copy remains on fiie at Vought. ' 




Utilization' bf Data 



The accumulated ^ubjective critiques of the simulator 
performance a/{d the training evaluations obtained from the 
student pilots and inputs from IPs are used^botli by TAG and 
Vought in evaluating potential improvements in the simula- 
tor and simulator training. 

. '23 < 



1^ The' objective measures of student, pilot performance 
^areiused in- obtainin3\^elfGSI to, represent a measure ofV 
"•relati^ve proficiency in ^ir- to- air .combat in the simulator. 



\Stud6nt pilot bacjcgrc^und "data are used to subjeptively 
-correlatie a pi'i^ot's, ^xpect^d leve-l'. ot^prof ici'ency with -^^-^ 
measured by thfe GSI. - " ^ " ' ^ ' f '/ 
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Experiment Controls y. ^ . ^ J 

The data werjS collected for a. sample of 90 subjects 
during the perigd of this study/ under concisely defined 
controlled conditions. The' study was unique in the sense 
that the data had to be collented^within ^nd from the opera- 
tional trairiiiig en virbnment. The collection, of d'pita under 
these conditions also had to be made' on a minimum interface 
and non-interfereAce* basis with the ongoing TAG ACES I ^ 
training program. This reqjuirement precluded the applica- 
tion of expeyrim^^ntal. controls in a classical sense, , as fqund 
in a laboratory experiment. As a result^ other methods of 
control were developed to function within the restrictions 
imposed to provide some assurance as to the fidelity of the 
dat-a collected and to minimize the effect of *undesired 
variables. 'This was accomplished by briefing each new Chief 
IP (CIP) *as to the mandatory adherence by IPs and students 
to the approved TAC ACES I Trainihg Syllabus. A form was 
developed and completed after each training^ class; certify- 
ing to the adherence to the TAC 3PES I Training Syllabus, 
fidelity bf the air combat simulator performance, cind per- 
formance accuracy of the software and computer hardware. 
Data collected from TAC ACES I students prior to this study ^ 
did not have these .controls<^ • ^ ^ 



T^e TAC>ACES; I' ^students in the study were not aware tyf 
the^GSI Validation 'gtudy and the purposes^ of data collection 
Individual pi lo^ performance data were co^llected on 'MQnday *• 
and Friday o£ the training week and , during the "Tuifkeyir* 
Shoot** eiimin^tioh' contest, after cdrapletioh of the. fonna.Iv 
training program. In addition; performance data were 
collected for four of the 12 classes on Wednesday of the \ 
training week. The students were alsd required to complete 
a bagkgrt^und questi^onnaire - and an end-of-course critique. * 
T^e existing questionnaires \^ere modified to* obtain age 
group and combat ^experience data. The Chief Instructor ' 
Pilots (CIPs) /fpr the TAG ACES program were required to pre- 
dict each student's performance dn the turkey .shoot contest. 
As each class completed the formal training program, the.CIP 
was required to rank-order that class^bf students as to 
their perceived? standing at the completion of the turkey 
shoot' elimination contest. Simulation or other training 
syllabus anomalies were- also recorded as a part of the data 
collection task to aid in the identif icfation of outliers in 
the data sets. A 

All of .the student pilot performance data were recorded 
if • '» 

on magnetic tape. All other data from students* background, 
course critiques, and CIP rankings were recorded on forms 
adapted to or 'g^erated for th*e s\udy^ In addition, afl of 
the stud^t pilot performance data were produced on hard 
copy pirintouts for verification and preliminaip/ analyses. 

The forms developed a<fa used in the study are included 
in Appendix B. The TAG ACES J( Training Syllabus and the 
turkey shoo''t. competition rules are included in Appendix C. " 
Mathematical descriptions of the scoring computations for' 
each weapon Simulated in the study have been submitted to 
the Flying Training Division of ^he Air Force Human Resour- 
c^^Laboratory . . . ^ 

✓ : 2S 





II. OBJECTIVES 



■ •"■ , V ~ I , ■■ ■ • • ' . •• ' V ' ' 'v' • 

- ' Tne' scop.e of thi^'. iTivestigatibri ■ IS limited, to the 

optimization and Validation of, the GST gystem. The 'primary 

product is an asses^ent- of the. capabitities/. and limitations 

' of the GSX scores -as indicators of. pilot Air Combat Maneu- ^ 

vering (ACM) skill and the determination of the utility of 

GSI scores as predjLctors of pilo^t performance in free-engage 

ment turkey shoot competitioji. 

Derivation of Optimal Models 
> 

The empiric^ally derived GSI was statistically valida- 
ted to its 'predictive capability by the use of . statistical 
analysis techniques *%n ^improved GSI predictor ^sing the 
four subjectively selected parameters i of the empirical GSI 
was obtai*ned by discriminant analyses. ^A further' improved 
GSI predidtor was derived from the expa-nded list, of availa- 
ble candidate predictor variables and variable selection 
techniques. These improved predictors were validartzed with 
data acquired #rom classes outside the experiment. Confi- 
dence intervals on the predictors were provided. Further, 
standardized discriminant functions were provided to identi- 
fy the relative contribution 9f each parameter in the de- 

rived predictor equation (s). Student pilot background and 

i 

subjective data obtained from questionnaire form^ were input 
with objective data. to obtain optimal, predictor^ models . 



' Comparison with Expert Opinion < / 

Subjective rankings of student pilots were obtained 
froii^' Instructor Pilots and compared to the derived, GSx' 
predictors and. the.. actual pilot rankings cfetained from • ' 
' tutkey shoot/results. ; These interrelationships were des- [ 
^ cribed .t^rot«B|hJJie , use of correlation and variaric^e/co- , 
variance mat^rixesl ' ^ " ' 

r Corr:elation With Previous Data *^ V 

■ < ' •* » 

' > - Da:t.a from classes undergoing training prior to thi^^ 

, - , experimental study: were usegf^ on a random selected basis ta 
/ / obtain measures of GSI prediction accuracies. These in- 

vest igatiph6 are Jiecessarily limited to the GSI as deter- » 
mined frbm the four subjectively selected parameters, since, 
other objective data were not ori file. 

A • • ■ . 

^ ■ ^ ■' ' J 

. Reliability of GSI Scores / . ^ ■ . " 

- . * 'The reliability of the GSI was determined by calculat- 
» ing^ confidence in tervals.^f predictions of turkey shoot rank 

and corresponding confidence levels of the degree of -ce^;^ 
' tainty of the predicted value. 

- ^; *Edumetric ^d Psychom,etfric Measurement 

■ * : . ^; • X . . . 

A measure of • learning effects, was obtained by statis- 
^ tically analyzing data from four classes specifically struc- 

y ^ tured to obtain tihree scoring periods for each student pilot. 
Measures of individual and group learning were statistically 
derived as a function of time in training. These learning 
rates were compared to student pilot performance .data. 



A 



EKLC 



22 

27 



1 11.^ )&NALYSES 




The GSI Score was computed from data acquired during 
the TAG. ACES I ^training of each class, normally on Mondty 
-and Friday.. During the GSI Validation Study,; a third ^et 
of GSI -data was collected on Wednesday Ifor^ four of the 12 
classes involved. GSI data are recorded nominally against 
five canned targejts; generally, two of the five are cine- 
track and the remaining three are head-on. 
The equation defining GSI is, 

GSI = 4.6 (TOrMILERR) + 0 . 86 (PANG) + (O/D-35) +0 . 5 (ISO-TTFK) ^ 
where; • ^ 

MIL ERI^- aver'age mil error over two cinetrack 

runs while R < 3,000 ft. 
PANG average percentage of engagement time in 

pointing angle advantage*, R < 3000 ft.\ 
•* over two cinetrack rxins., 
0/D - average ratio of offensive to defensive 
- ^lime against the head-on targets. Offen- 
sive time is the time this target ai^rcraft 
is in the front hemisphere of , the piloted 
. aircraft. ' . 

TTFK /' average time to first kill (second^) from 
♦ ( beginning of fun until student achieves ' 

first kill against head-on targets with gxin 
or heat missile; \ 
The GSI Score itself is intended, to have a possible, 
range between zero and 1,000. Also, each of the four compo- 
nent scores was originally intended to contribute equally to 
the index itself. Scaling factors were adjusted from time 
to time as experience was gained and when an adjustment was 
considered appropriate. The equation for GSI given above 
contains the scaling factors used over the data collection 
period of this study. MIL ERR, PANG, 0/D, artd TTFK are 



referred to as ttxe GSI component scored or component 
^ariables m this report. 

S^tistical Analysis of GSI Data 

The statistical analysis of the basic Moh^y afjd Fri- 
day GSI. scores and the four GSI component scores '^collect^d 
over the -l2-class experimental pelriod is presented in this 
section. * V 

Histograms of the GSI scores and the four GSI compo- 
nent variables (part-scores) are provided in Figures 2 
through 6. These show the general distributional grapes of 
each- variable. The histograms for Monday and Friday for 
each score are provided on the same page to ^facilitate 
^^^isual comparison. In general, the distributions improve 
from Monday to Friday (increase or decrease as appropriate) 
curd the sample standard deviations become smaller. / 

Scatter diagrams for" GSI and GSI gomponent variables 
^or both Monday an<3 Fridd^^re presented in Figur^ 7 
through 11« The Y-variable used to construct these scatter 
diagrams ig turkey ^hoot rank. Turkey shoot winners are 
ranked one, runners-up are ranked two, third eliminators- 
always receive a ranK of 5.5, and first eliminators are 
generally ranked 7.5. A visual examination of these scatter 
diagrams reveal,s no apparent trends .4^ 

Early in the analysis, a second candidate Y-variable 
was considered to be of possible interest. This was frac- 
tional wins, defined as^ the ratio of turkey shoot wins to 
the total number of engagements for a given student as indi- 
cated on thjB double elimination tree used to score the 
turkey shoots. Correlation coerf ::ients of the four GSI 
component variabi'es to turkey shoot rank and f ractional/wins 
for both Monday and Friday data are shown in Table 1. The 
presentation is constructed so that the correlation 
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TABLE 1 - GSI CORREiLATION COEFFICIENTS 



MONDAY 



T.S: AVG.Mli; 
RANK ERR % E'ANG 



% OFF 
TIME 




FRIDAY 



MONDAY 



FRACT. WIN 
AVG.MIL. ERR. 

% PANG 

% OFF TIME 

TTFK 



FRACT. AV9,,MIL ' ^- ^. qff" 

■ WIN ■ ERR % PANG ' - TIME 




-.1355 .1759 .0261 

1 -.0891 -.1915" 

-.3071'?' r^l .2107 

-.0951 .0007 ^^1 

.0559 -.1557 -.6052 

FRIDAY ~~- ■ 



35 



4 0 



coef/icients fop: Monday data are shown above tlie main dia- 
gonal of each matrix ai>d for Friday data are below the main 
diagonal. As can be seen, relatively strong -correlations 
.6xist among the , component variables indicating non-zero co- 
variances and thus lack of independence, i.e., possible 
significant multicolinearities . ^ Correlations between the 
component variables and turkey shoot rank and fractional 
wins are also seen 4:o be very^weak. Various regression 
analyses using appropriate variable aelectij&i techniques 
and ridge regression were also conducted as part of this' 
study^ Predictive capabilities of^ these regression models 
. were found to be very poor.. Th±s is what might be 'expected 
in view of the scatter diagrams provided. 

In^n attempt to determine significant sources 'of 
variation within the data, five three-way analyses of • . 
variance were 'conducted for GSI and the four component 
variables .' The three sources of variation investigated, 
wete - - . * , 

(a) variation between days, (Monday and Friday) , 
: (b) variation between turkey shoot ranks, and 

(c) yariation between the cl^asses which^ contained 
eight students. 

Table 2 shows the results of the Analysis of the GSI 
scores. It was found that very significant differences 
exist between Monday and Fridfeiy GSI scores (The risk of 
error in saying a significant difference exists when in fact 
it does not is less than one percent) , implying, -of course, 
that if GSI measures group /le arning , a significant increase 
occurs over the five-day ' class period.^ This is discussed in 
detail in the section on e.dumetrics. The other significant 
source of variation (also significant at the-one percent 
level) is between classes. It was preferred that significant 
differences betW^een clasiSes would not occur, as this 



.TABLE 2 -■ ANALYSll-OF VARIANCE - CfSI SCORES 



■ SOURCE OF 
VARIATION 


SUM OF SQ. 


DF ' 


MEAN SQ. 


F -TEST . 


BETWEEN 
DAYS 


997,335 


1 


997,335 


51.1**- 


BETWEEN 
RANKS 


58/630 


3 


19,543 


, 1.00 


BETWEEN 
CLASSES 


655,204 


8 


.81,900 


4.20** 


RESIDUAL 


2,557,437 


131 


19,522 




TOTAL 


. 4, 26 8', 606 


' 143 







significant at 5%- level, 
^significant at 1% level. 
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could tend to mask differences between ranks, as exhibited 
in the data,^ if they really existed. Conversely, it was 
desired that significant differences between GSI scores by 
rank should occur. These differences, did not occur, and 
this provides evidence as to why the initial GSI score is a 
relatively poor predictor of turkey shoot ranki Figure 7, 
which shows scatter diagrams of GSI scores versus turkey 
shoot rank, provides graphic evidence as to why significant ' 
differences between GSI Score and rank do not exist, or at 
least> they cannot be detected from these data. 

Tables . 3 and 4^ present the three-way analysis of ■ 
variance tables^ for the GSI component variables. For the 
component, variable average mil error, significant differ- 
ences between ranks appear to exist at the .l percent confi- . 
dence level, but no difference is evident between days. A 
difference is detectable between classes at the 5 percent 
level. 

For the component variable percent PANG, significant 
differences are .evident at the 1 percent level. There is no 
evidence of significance for variation between ranks. For 
the component variable, offensive time, significance be- 
tween days are detected at the 1 percent level. No differ- 
ences appear 'to exist ^between ranks or^classes . For the 
component variable TTFK, significant differences are de- 
tected at the 5 perceht level between days and betwe.en 
ranks. Differences are not "evident betweer classes. ^ Table 
5 summarizes the finding of the analyses of variance per- 
fonned of the four GSI component variables. 



TABLE 3 - GSl ' COMPONENT ANALYSIS OF, VARIANCE 
ANALYSIS OF VARIANCE - AVERAGE MIL ERROR 



SOURCE OF 
VARIATION 


SUM OF SQ. 


DF . 


MEAN SQ. 


F TEST 


BETWEEN - 
DAYS 


■ y 

152.11 


1 


'152.11 


.51 


■ * 

BETWEEN 
RANKS 


4,567.06 


3 


1,522.35 


; 

5. 15** 


BETWEEN 
CLASSES 


5,568.72 


■ 8 


696. -09 


2.35*, 


BESIDUAL 


38,764.33 ; , 


131 


^ 295.91 




TOTAL 


•49,052.22 


143 







ANALYSIS OF VARIANCE % PANG 



SOURCE OF 
VARIATION 


SUM OF S^. 


DF 


MEAN SQ. , 


F TEST 


BETWEEN 
DAYS 


v^, 871.1^ 


1 


2,871.17 


24.5** 


BETWEEN 
RANKS 


356.08 




• • 

118.69 


1.01 


BETWEEN 
CLASSES * 


4,114.25 i 


8 


• 514.28' 


1 

4.38** 


RESIDUAL 


15,371. 44 


131 


; 117.34 




TOTAL 


22,712.94 


143 


0 





* significant at 5% level- 
** significant at 1% leve^r 



TABLE 4 - GSI COMPONENT ANALYSIS OF VARIANCE 



ANALYSIS OF VARIANCE - % OFFENSIVE' TIME 



SOURCE OF 
VARIATION 


SUM OF SQ. 


DF 


MEAN SQ. 


F TEST 


BETWEEN 
DAYS 


6,696.69 


1 


6,696.69 


47.2** 


BETWEEN 
RANKS 


274.25 


3 


91.42 


.64 


BETWEEN 
CLASSES 


1,332.47 


8 


166.^6 


1.17 


RES t DUAL 


18,600.56 


13V 


141.99 




TOTAL . 


26,903.97 . 


143 







ANALYSIS OF VARIANCE - TIME TO FIRST KILL 



.SOURCE OF 
VARIATION 


SUM OF SjQ. 


DF 


MEAN SQ^ / 


F TEST^^ 


BETWEEN 
DAYS 


19^113.07 


1 


19,113.07/ 


2 3.2** 


BETWEEN 
RANKS 


13,215.75 


. 3 


I ^ 

4,405.25 


5.35** 


BETWEEN 
CLASSES 


10,873.01 


8 


_1^5Kl3 


1.65 


RESIDUAL 


107,942.50 


131 


^ V 

823.99 




r 

TOTAL 


151,144.33 


143 




— 



* signific^t at 5% level 
** significant at 1% level 



TABLE 5 - SUMMARY OF RESULTS OF ANAL^^S OF 

VARIANCE OF GSI COMPONENT VARIABLES 



SOURCE OF - « . • 

VARIATION MIL ERR % PANG % 0/^ TTFK 

BETWEEN 'days I [ ** 

BETWEEN RANKS - *y , _ _ ** 

BETWEEN CLASSES ^ * ** 

- . . . T ' 

* significant at' 5% level ■ i 

** significant at 1% level 

A ^Comparison of the GSI Predictor 

This section presents a comparison of the best predic- 
tor using the GSI Score as defihed||5 the beginning of the 
study with random selection and with CIP predictions (CIPPs) 
made jiist prior to the turkey , shoot/ competition. Compari- 
sons were made at four levpFs of detail as to the outcome 
of the turkey shoof (These levels of detail are carried 
throughout the remainder of the study) . The four levels are 
defined as follows: 

1. Four, Groups - Proper placement into the proper 

turkey shoot quartile, i.e., 1 or 
.2 in the first group, 3-4 in the 

second group, 5-6 in the third group 
and 7-8 in the fourth group. ' 

2. Upper Half \ Proper placement of students in the 
of Class - ) ^ ^ 

top four turkey, shoot ranks in those 

ranks,, i.e., 1, 2, 3, 3.5 or 4 in 
these ranks. « 

3. Winner and - Prbper placement of the winner oi 
Runner-Up - ' 

runner-up in the winner/runner-up 
V group. ' 



4, Wim^eo: - Proper identification of the - 

\ actual turkey shoot winner. . 

The results of this cbmparison are provided in Table 6\ 
Note that CIPPs were not made for^the first few classes 
of the experiment; thus^ only 67 out of a possible 90 CIPPs 
welvifuade* The randon^ selection probabilities were deter- 
.mined under the assumption of independent random assignment 
of students to turkey shoot position. For . example, these 
are eight possible^ssignments of outcome to the turkey 
shoot position* ^ne of these positions is the Winner posi- 
tion; another is tiJie runner-up position; two are third 
eliminator positions, etc. Thus/ the probability that ^ 

given student will be assigned the winner position, given 

. ' ■ * 

^Uxa -t his assignment is at random and independent of all 

■■ ■ \ u ' . 

other assignments, is one out of eight or 12/5 percent. 

Similarly , the grouping being considered iSv winner and/ or 
runner-up, there are two out of. eight possible assignments 
in this group* Therefore, under the same assumption, the 
probability that a given student will be assigned to the 
winner and runner-up grouping is two out of eight or 25 
percent • Similar logic is used in determining the probabi- 
lities associated with the random assignments to the other 
two groups • 

Four entries^ are provided for CIPP and GSI ranking 
predictors for each of the four groupings* These provide 
basic data .on the actual predictions* For example, for 
CIPP aiid the "four groups" grouping, the CIPs properly 
placed 21 out of 67 predictions in the correct groupings 
(1-2, 3-4, 5-6, or 7-8) ; thus, 21 of 67 or 31* 3 percent 
were- Correctly classified* Ninety-five percent confi^ltence 
limits were calculated using these data and were determined- 



.TABLE 6 r- , 

A COMPARISON OF FRIDAY GSI RANK PREDICTIONS 
WITH CHIEF INSTRlJCTOR PILOT (CIPP) AND 

' RANDOM SELECTION 



0 

GROUPINGS 


IK 


RANDOM 
SELECT. 


CIPP 


GSI 
" RANKING 
(FRI. SCORE) 


FOUR GROUPS 
(1-2,3-4, 
5-6,7-8) 

• 


1 

. NO. CORRECT 
PREDICT. 

. TOTAL NO. 
PREDICT. 

. % CORRECT 
PREDICT. 

. 95% CONFI- 
DENCE INT. 


25% 


21 
' 67 • 
31.3% ' 
" 20.2-42.5 


26 « 
90 
28.9% 
19.5-38.3 


t 

UPPER HALF 
OF 'CLASS 
(1,2,3-4) 


/ NO. CORRECT 
PREDICT. 

. TOTAL NO. 

PREDICT.' 
,. % CORRECT ' 
PREDICT. 

. 95% CONFI- 
DENCE INT. 


50%-^ 


24 
34 

70. k% 

55.2-8>.9 


27 ' 

y 46 

58.7% 
44.5-72.9 ' 





. NO. CORRECT 




6 


9 




PREDICT. 








WINNER & 
feUNNER-UP 
(1, 2) 


. TOTAL NO./ 
PREDICT. / 
% CORREOT 


25% 


17 
35.3% 


23 
39.1% 


PREDI^Hr; 








• 


. 95% CONFI- 




12.6-58.0 


19.2-59.1 




DENCE INT . 










. NO. CORRECT 




1 


3 




PREDICT. 










. . TOTAL NO. 




9 


12 


WINNER 
(1) 


PREDICT. 
. % CORRECT 


'12.5% 


11.1% 


25.0% 


PREDICT. 










. 95% CONFI- 




0-31.6 


0-;4af';5 




DENCE INT. 
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. to be 20,2 pe^ent and 42. S percerit"''. Thus, over the long 
ruH/ 95 percent of- the CIPPs can be 6xpec^ed to be betWefen 
20.2 and 42.5- percent correct. Similar information -is pro- 
vided for the other CIPP and the GSI rankirig predictors . 

Each CIPP and GSI ranking prediction! was subjected 
to a t^st of the hypothesis that it isjequal to or^better 
than r'andom, selection^'. The ClPP^^^Ul^the upper half of the 
turkey shoot was found to be significantly better than ran- 
dom s^lecti-on at' the 5 percent confidence level. The GSI 
ranking predictor was found tp be significantly better t^an 
random selection for winner and runner-up also at the 5 
percisnt confidence level. All other predictions were found 
not to be significantly different from random prec^iction at 
the 5 percent level. Table 7 provides the levels of signi- 
ficance at which differences would Be assumed to exist. - 

. "C^ABLE^? - ip^PROXIMATE RISK L'feVEL I-.T WHICH 

DIFFERENCES -CAN BE ASSUMED TO EXIST 

GROUPINGS CIPP ' GSI RANKING 

" ' " "" ^ ' ' ■ ^/ . • 

y 

FOUR GROUPS 15% 18% 

UPPER HALF 5% ^ 13% 

WINNER & RUNNER-UP - 26% 5% 

WINNER 36% 20% 



1 



Ostle & Mensing. Statistics in research ^ (3rd ed. ) . Ames: 
Iowa State University Press, 1975, 100-101, 

ostle & Mensing. Statistics in research , (3rd ed.). Ames: 
Iowa State University Press, 1975, 129-133. 



Thus, to this point*^ in the ^alysis, it can be con- 
cluded that GIPPs can classify students ^s to whether or . 
-not they will fini^jfin the upper half of the turkey shoot 
witlj about 55r^o 86 percent accuracy while a . simple GSI 
ranking scheme can correctly predigt turkey shoot winner'^/;. : 
and runner-up classification . about 39 percent of the time^. . ^ 
For other predictioris investigated, the two predictors, 
appear to <^e no better than random selection. The data in 
Table 6 will be carried forward for comparison with more' 
sophisticated predictors developed from the expanded^ data 
sets acquired from the master data base, and through the use 
of discriminant analysis.' 

The Discriminant Analysis - A Discussion of the Analysis 
. Perfo rmed ' ' ~ — 

* ■ ^ . 

The GSI scores, the GSI component variables, the expan- 
ded set of candidate predictor variables, and the demogra- 
phic data were slabjected ^to a series of discriminant analy- 
ses using the sub-program DISCRIMINANT available as ^^a^t of 
the SPSS package"^. i'he capabilities of this program were 
useful in the development of predictor equations from the 
available data. The purpose of this analysis was fio build 
optimal prediction models- which predict "turkey shoot" rank 
from data collected during the 12 specified TAG ACES I 
classes.' The models derived used the Wilks' Lambda variable 
selection criteria^ to select the best candidate predictor 
variables from those available. The models derived are 
optimal within the constraints of the analysis but are not 
necessarily maximal. A maximal predictor model could only 
be achieved if all possible models were considered. 



■^Nie. Statistical Package for the Social Sciences (SPSS) y 
(2nd ed.) . New York: licGraw Hill, 1975, 434-462. 
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Discriminant analysis begins with the desire to sta- 
tistically distinguish between two or more defined groups 
using information available from saiT¥>le data. It was de- 
sired to predict turkey shoot winnera using data collected 
by the simulator computer jfrom each student during the nor- 
mal course of his trainitw anrd^lso from questionnaires.^^ 
The groupp.ngs od interest were defined from turkey shoot 
> rah<j^ In a normal\clas^ of eight student: pilots, there 
ai^e always at least rive distinguishable" turkey shoot 
groupings. These are in order from most favorable to 
least favorable outcome: winner^ (i-ltf runner-up (1) , third 
eliminators (2) , second eliminators (2) , and first elimi- 
l\nators (2) . « * • 

f The primary objective of the analysis, was to develop 

'^predictor algorithms for turkey shoot ^winners ; therefore, 
the groupings considered were structured to investigate the 
. level of detail at which winners could be predicted . from 
available data; Winners "can be de:^ined in several^ ways . 
One winner class is the absolute winner *or undefeated stu- 
dent in the turkey shoot A second winner, class is the 
winner and runner-oip. This grouping scheme was used with 
some limited success in earlier Vought investigations, which 
employed Friday GSI as the, predictor vari^le. A third 
levej. of detail is the upper. half of a c^ass as determined 

i 

by the turkey shoot competition. /In all, four different 
grouping schemes were defined and investigated. These 
are as follows: * _/ . > 

1. Winners (Group I) versus all others (Group II) * 

2. Winners and ruijners-up together (Group I) versus 
all others (Group II) 

3. The upper half of the cl^ss (Group I; winners, ' 
Vunnersnup-, an'd/third elijninators) versus the 
lower half of the j^lass (Group II: second 



eliminators .and firs^' eliminators). 

4. Four Groupings (Group I: winners and. runner s- 
up,%. Group II: third eliminators; Group III: 
second eliminators; Group IV: first elimina- 
tors ) . . 

The analysis was conducted in four Mparts, each part * 
being defined by the candidate .'predictor variable set to 
be used. The first analysis used only Monday and .Friday 
GS I scores as candidate predictor variables. This analy- . 
sis provided a measure of the- best prediction capability 
of the GSI itself. Both the Monday and the Friday GS I scores 
were presented to DISCRIM as candidate predictor variables 
Thus, DISCRIM was able to select one, the other, or both 
GSI scores. As it turned out in the three winner group-- 
' ings investiga^ted.in the first analysis, both GSI scores 
were always included. The predictive capabilities - 
determined her^were then used as the baseline,, or basis 
of comparison, for th* other three analyses which folloWd. 

The discriminant artalysis considers more than just 
correct classification into the desired group." Two ■ groups 
are defined, one group including the winners, and the 
other group including the non-winners. It is possible to ^ 
correctly classify most of the true winners but incorrectly , 
classify somis relatively large number of ^non-winners as 
winners. It must be decided how many non-winners can be 
accented in the winner group. This study found that by 
using^-indicators more complex than the GSI Score itself, ^ 
it was possible not only to correctly classify "winners" ^ 
a fairly large ^.percent of the time^ but also to greatly^ f 
reduce the classification of non-winners into the winner ) 
group . : 

The analysis began with the empirically determined 
GSI scores as predictor variabiles.. In the second analysis. 



V . - 

the four componenrt ^variables {or' part scores) from which 
GSI is calculated were used, instead of the GSI total scores. 
. The E|ISCRIM program was then allowed to select from these 
eight component variables (four ^or Monday and four for Fri- 
day) the best predictor* variables for each of the four 
classification schemes. The eight variables'^are defined in 
Table 8 which shows thSat DI SCRIM was selective and never 
used all available data to define the optimal prediction 
(classification) equations. ^" 

Results of the Discriminant Analysis 

The results of the/£our discgiminarrt- analyses are pre- 
sented. Five pieces of information are priovided for each 
discriminant grouping scheme : ^ V 

1. A tabulation of group predicted membership versus 
actual group membership, ^using the 12-class sample cofisi- 
dered in the study. 

2. The basic optimal classification functions deter- 
mined by the discriminant program. These are presented in 
tabular form. The classification functions are used to pre- 
dict group membership. There is one classification function 
for each defined discriminant group. ^Tojclassify a given . 
sample (case), the value (score) for each* classification 
function is calculated. The sample (case) is then classi- 
fied into the group for which the ^classification .function • 
provides the highest score. ^ 

3. Standardized Discriiliinant Function (s) -- In this 
study/ there is always on^ less discriminant function than 
the number of groups- defined. * In general, the discriminant 
functions can be thought of a^^he axes of a geometric 
space, and thus can be used ti'study the spatial 



TABLE 8 - MONDAY AND FRIDAY GS I COMPONENT VARIABLES AND 
VARIABLE SELECTION BY DISJIRIMINANT GROUP 





( 


-L'ROL'P I - Winners; GROUP II - (^hers 




i 


GROUP I - Winners & ' Runners-Up ; GROUP 11 - 
Others ' ^ - . 


VAR.. 

DESIG. 




1 


GROUP I - Winners , R. a. ; -6. 3rd Elim. ; 
GROUP II Others 




GP. I - Win. &'R.U.; GP . II -.3rd Elim., 
GP. Ill - 2nd Elim.; GP..IV - ist Elim. 


VARIABLE DEFINITION . 


XI 
X2 
X3 
X4 


■ 


V 

X 

y 


X 


X- 

X 


AirEYilKGE MIL .ERROR FOR FRIDAY 
PERCENT TIME IN PANG FOR FRIDAY- 
PERCENT OFFENSIVE TIME FOR FRIDAY 

TIMj^ TO FIRST KILL 'ON FRIDAY (SECcTnDS) ' 
^ ■ . ' ■ ■ • . 


X6' 

X7 


X 

t 


X 

i 
1 


X 


r 

X 


-AVERAGE MIL ERROR FOi- MONDAY ■ ' 
PERCENT TI-ME IN P.A-NG FOR MONDAY ' ' 
PERCENT OFFENSIVE TIMJ?: FOR MONDAY 
TIME TO FIRST KILL ON MONDAY (SECONDS) " 

■ ■ i 



> ^ ■ 




i 




relationships among the groups • The standardized dis- ' - 
.'criminant ftinctions perform the same general functions as, 
the standardized (beta) coefficients in regression analy^ 
sis. These functions- provide ^ easy reference as. to the 
relative contribution of each of the selected discriminant 
predictor variables, 

4. - Unstandardized Discriminant Functions — The un- 
Standardized discriminant functions, like the standardi^d, 

' are useful in the descifiptive analysis of spatial relation- 
ships among the groups, - ^ 

5. .Canonical Correlation Coefficients of the Discri- 
minant Function (s) — The canonical correration coefficient 
jg5r.ovides >n indication of the relative cap'ability of the ' 
associated discriminant function to. .separate data into 
correct groups, ^.a value of one indicates ' perfect group 

separation capability; a value^of zero -indicates total in- 

■ ' • ' . ' ' ' ■ , . ■ . • ' 

ability to separate grou^ss^V '<^ju * U ' 

. ' ' . -^'^ - ^ . ' . ' -^^ 

ThB Firsts Disgrimiriant Analysis - Assejssment of the GSI 
Scores as Turkey Shoot Placement Predictors 

' . ■ ' . "^"^"^^^ ■ 

The ri^palts of the fir^t discriminant analysis are 
presented inl^ables 9, 10^ and 11, where Monday and Friday 
GSI scores are, the predigtor variables. While, in general, 
mfrabeisip^ of .the first group are- correctly classified on th§ . 
bi^er^ of '60, percent of the^ time, many^ non- first group stu:^^^ 
dents are classified, incorrectly in the first group. The^^ 
lack of discriminant power is evidenced by the low vaLues^-; 
of the canonical correlation coefficients of the resc^ctive; 
discriminant functions, i.e. , betwei^n 0 . 120. and 0.'218. . ^ ' 
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TABLE 9 - GSI TURKEY . SJIOOT WINNER PREDICTIONS 



PREDICTOR VARIABLES: Monday and Friday GSI Scores 



1 ACTUAL CaROOP 
1 MEMBERSHIP 


NO. OF 
CASES 


HtEDIClED GROUP MEJffiEftSHIP 


■ GROUP I 


GROUP II 


1 Turkey Shoot 

1 Winners ' GPI 


12 


66.7% 


4 

33. 3% 


1 Turkey Shoot Non- 




34 


44 


Iwinners (Others) GPU 


■ 78 \ 




4;3.6% 


56.4% 



57, 



CLASSIFICATI ON Vu TTCTION 
-COEFFICIENTS 



FGSI 



MGSI 



CONSTANT 



0.03907 



0.00400 



-13i77014 



0.03594 



0.00581 



-12.55049 



DISCRIMIKAN T- FU NCTION 
COEFFICIENTS 



-p^9_6118 
0.75773 



-0.00757 
0.00437 



2. 80178 



CANONICAL CORRELATION OF DISCRIMINANT FUNCTION IS 0.140 
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TABLE 10 - GSI TURKEY SHOOT WINNER AND 
. ^ RUNNER-UP PREDICTIONS 



HffiPICTOR VAHIABLES: Monday and Friday GSI Scores 


ACTUAL GROUP 
MEMBERSHIP 


. NO. CF 
CASES 


. HffiDICTED GROUP MEMBERSHIP 


^OlODP I 


GROUP II 


Turkey Shoot - -. / 
Winners & Runners GPI 


23 


'14 
60.9% 


9 

39.1% 


T^ird, Second, and 
First Elimina- " GPU 
tors (Others) 


^ 6 7'. 


33 

49.3% ' 


34 
50 .7% 


5.^.3 i OF CASES WERE CORRECTLY GROUPED 




\ 

VARIABLE 


1 ^ CXiAoSIFICATI 
1 OOEFT.I( 


[oiT rnfCTioN 

JIERTS 


DISCRIMinAI 
COEFFK 

STANDARDIZED . 


fT FIWCTIGN 
7IENTS 


FGSI ' ; 


Q>. 0 3804 


0.0396 


0.96359 




0.00759 
0..'00055 


MGSI" 


o.oosge" 


0. '00581 


0Td9 485 . 


CONSTANT 


-13.85377 - 


-12.45 815 




-5.02757 


































* ■ 


































§ ^ 




• 


















■ < tiV 






1 




1 "" 
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CANCariCAL CORRELATION OF DISCRIMIRAlfT FTWCTION IS 0 . 120 





TABLE 11 - GSI TURK-Ey SHOOT WINNERS, RUNNERS-UP AND VhIRD 
. ELIMINATOR PREDICTIONS (CLASS UPPER HALF) 



HffiDICTCK VARIABLES: Monday and Frida^ GSI Scores 



J ACTOAL GEROOP - , 
1 MEMBERSHIP 


NO. CF 
CASES 


, ' HffiDICTED GROUP MEMBERSHIP 


GEROUP I 


GROUP II 


1 T.§. Winners, 

1 Runners-Up and GPI 

1 Third Eliminators 


46 


27 
58.7% 


19 
41.3% 


j T.S . Second and 

1 First Eliminators GPU 


44 


" ' 19 , 
• 43.2% > 


23 

, 56.8%^, ' 



57.6 ^ OF CASES WERE CORRECTLY GROUPED 



VARIABLE 



CLASSIFICATICW FOWCTION 

coEPFiciErrrs 



DISCRIMniANT FUNCTION 
COEFFICIEnTS 



FGSI 



0.03764 



MGSI 



0 .00759 



CONSTANT 1-14.06189 



<3R0UP IT I STAimAPDTTRn 



' 0.03587 



0.00574 



0.. 49 739 
0 .71-39 8, 



-1^.11746 



0.00392 
0.00412 



-4.32199 



CXRCtTKAfj CORRELATION OF DISCRIMINANT FUNCTION IS 0.218 ' 



53 



The Second Discriminant Analysis - StatisJ^cal De^ivl 
tion of an Optimal Four Parameter Predictor - Derives^ 
Optimal Predictors Using the Same Four Parameters of the 
Empirically Derived GSI Scores a 

The results of the second discriminant analysis, are ^ 
presented in Tables 12, 13, 14, "^and 15. In this anaiysis^^ 
the eight GSI .compoheint^ variables (four for Monday GSI 
component scores and four for Friday GSI compcfnent scores) 
are used as candidate predictor variables (Table 8). The 
table for each gi^oup definition 'indicates -the variables 
selected by DISCRrM. For example , X3 and X8 (Percent 
Offensive Time for Friday and Time to First Kill (TTFK) for 
Monday, respectively) were selected by DISCRIM for inclusion 
in the analysis where the d2 turkey shoot winners comprise 
the top discriminant group. The predictive^capabiliti^s of 
thi'S , analysis" appear to be marginally better than in the 
GST score amalysis. The second analysis also investigated 
four groupings '(quartile ranking) (Table 15). ^he standard- 
ized and unstandardized discriminant function Coefficients 
are also presented in Table 15. 

The Third Discriminant Analysis - Statistical Derivial^ion 
of Turkey Shoot Placement Predictor from an Expanded 
Objective Data Set. " ~" '. '. 

Th^ results of the third discriminant ' analysis are pre- 
sented in Tables 16, 17, 18, and IS. Candidate predictor 
variables were developed from the complete objective data 
set collected during the Monday arid Friday GSI scoring 
session but; previously not analyzed. The table for eaphn 
group definition indicates the predictor variables selected 
for the ^iven grouping scheme. The expanded set' of candi- 
/dat;e variables and their def InitgLo'i^s a;re contained in Table 
20. The canonical correlations of the discriminant 
■ • . ' • • - ^t ^ ■ ^ ^' - ' 

. 4 54 ^ , 
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TABLE 12 - 



OPTIMAL FOUR PARAMETER TURJvtEV SHOOT WINNER 
PREDICTORS 




ERIC 
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TftBLE. -13 - OPTIMA! FOUR PARAMETER TURKEY SHOOT WINNER 
\ AND RUNNER-UP PREDICfOR ^ , • 



VARIABLES: Monday and Friday GSI Component 
Variables 



/^ACTOAL GROUP 
f MEMBEIlSHlP 


NO. OF 

CASES 


HiEDIClED GROUP MEMBEEISHIP 


^ GESOUP I 


GROUP II 


.^urkey Shoot 
^f^nners & Runners GPI 

-Up • 


. 23 


15. 
"65. "2% 


8 

•34. 8% 


Third, Second & , 
First EliminatprsGPn* 


67 


2y 

' 43.3% 


38 

. 56.7% 



, VARIABLE 

t 


1 CLASSIFICAT] 
I ' COEPFIC 


COW FOfGTION 
/lERTS 

GROUP 


DISCRIMIKAIi 
COEFTIC 

STANDABDIZED 


FT FIDTCTIOR- 
JIERTS 

unstahdabdized 


1^ 


u • J. / o U 7 




-0 . 74791 

-0.45291 


-0.04149 
-0.04032 . 


X2 * ' 


0. 42651 


0. 45655 


X4 


0.19877 


0.21484 


-0.51216 • 


-0.02157 


X5 


0.00750 


- 0.01261 


-0.42452 


-0.00686 


_ Constant 


-19.45735 


-23.34183 




5.38022 










































-le- 














































































t 







CANCmCAL CORRELATION OF DISCRDORANT FtWCTION IS 0.301 
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TABLE 14 - OPTIMAL FOUR PARAMETER TURKEY* SHOOT WINNER, 
' r RUlINErR-UP •& 3RD ELIMINATC^ PRED. .{UPPER HALF) 

HlEpiCTCffi VARlA^LESf Monday and Friday 'gs I Component 

Variables 




61.1 % OF CAfe WERE, CORRECTLY GROUPED 



VARIABLE 



X3 



X8 



JtASSlFICATIOW PWfCTIOT 
COEFFICIENTS 



I . 



2.23993 



0.04012 



2.20200 



0.61428 



0.05417 



DISCRBirNANT FUHCTION 
COEFFICIENTS 



STAlTOATmj pED 



_0_._37015 
-0.39683 



-0.61199 



IWSTAHDABDTZEnl 



0.04714 
-0.01671 



-0.01^45 



Constant, 



-lOQ.snmn 



-110.02942 



0.6469 2 



r 



4^ 



CAHOnCAL CORRELATION OF DISCRIMERANT FOTCTION IS 0.35 7 



TABLE 15 - OPTIMAL FOUR PARAMETEFl QUARTILE" RANK PRE- 
DICTORS 



ERIC 



ArTUAT^ GROUP 
MEMBERSHIP . 


NO. 

OF 


PREDICTED GROOP MEMBffe] 


RSHIP 


GP I 


•(5P II 


-GP JII 


GP IV 


Turkey Shoot ^ GP I 
V7irmers & Runners Dp 


23 


10 
43.5% 


5 

21.7% 


4 

17.4% 


4 

. 17.4% 


Turkey Shoot GP II 
Third Eliminators . 


\' 


4 

17.4% 


. 13 
56.5% 


.1 ' 
4.3% 


5- 
21^7% 


Turkey Shoot . '^"^"^ 
Second Eliminators 


23 


7 ' 
30.4%^ 


r 

K .4/ 
17.4% 


- 5 . 
21.^% 


7 

30.4% 


Turkey .Shoot '^^ 
F ir ^JlnPl imin ator s 


..-•2-1 


4 

, 19.. 0% 


3 

^tJ4^.3% 


28. ey 


8 

38.1% 



PREDICTOR VARIABLES; Monday ang Friday GSI Component 
-r Variables ; , ■ ' 



40.0% OF CASES WERE CORRECTLY GBOUPgD ' 



CLASS IFICATION FUNCTJ ONVcOEFFICiENXS 





1 GROUftf I 


' GROUP II' 


'GRj!)Ui> III 


GROUP IV 


XI 


0.14229 


0.18733 f 


^6.1652 3 


0.144-52 


X3 


, 1.26789 


1.27884, ■ 


" 1.2383:^ ^ 


1.17700 


X5 


0.01537 


0.02250 


■ 0.01595 


0.02.180 


X8 


0.10084 


0.08284 ■ 


0.107^8 


0.10920 . 


Cojistant 


-53.39603^ 


-53.99324 


-52.95479 


-4 8_.69345_ 














■ 




- / ' , 


























— — 




. • • ■ f. 
1 ! £ 


■ . ■ \ .. . ' 
















* ■ ^ 








I " ' 

— / — 

















































58 



6:^ 



X 



TABLE 15 (CONT.) 



1 •PKEDICT( 

-\ ■ 


DR VARIABLE SET; Monday and 

Variables 


Friday GSI Component 






DISCRIMINANT FUNCTION COEFFICIENTS 




IVARIABIiE 


STANDARDIZED 


UNSTANDARDI ZED- 




FCN . I 


FCN. II 


FCN. Ill 


FCN. I 


FCN. II 


FCN. III 


1 yi - ■ 




6 - 45.174 


-0 * 7049 4 






-0.03907 


1 X3 




-0.54261 


-0. 05719 ^ 


, -0 0 6 5rt7 


U » U O j7 X X 


-0 . OU / Zo 


X5- 


-0.10805 


0.60537 


0 . 49927 


-0 0017S 


n n n Q 7 Q 


u . uu bOo 


1 X8 . 


0.69680 


-0.14693 


-0.57239 


- U. 01487 


-0 . 00419 


- 0 * 0 1 h 9 


boNSTAi>JT 


— 





_ 


3.21251 


3. 93242 


<ij . u <ij «j «^ ^ 


















■X 








~/~~ 




CANONICAL 














PORREL. 


■ D.427. 


0.2 81 ' 


0.162 . 






















I* 


. , _ 1 






























































— II i 






1 

















































t 




»— — ^ 


— ^ 










'—^ 






















f ■ ■ - 


— 3 
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TABLE 16 - EXPANDED OBJECTIVE PARAMETERS TURKEY SHOOT 
WINNER PREDICTORS ^ ^ - 



EKLC 



HffiDICTOR VARIABLES: Expanded Data Set (Without DemographijC 

Data) 




WRIABLE 



CLASSIFICATION FUNCTION 
COEFFICIENTS 



GROUP T 



M8 



M12 



M16 



M29 



M32 



. 0 .00575 



,0 .86 869 



1.58034 



0 . 18453 



• Q . 0 2/9 2 8 



' Fll ' 



1. 3S 074 



F18 



0 .£(5870 



F22 



-0 . 10910 



F2 3 



0. 15750 



^^^^Z^722 - 



F2 7 



4.35721 



F2 9 



0 . 35 718 



CONjSTANT 



•118.97914 



0 . 00755 



0 . 82532 



1.19 862 



0 .23185 



0 .02361 



0 .61329 



0 .16967 



17 



0.09483 



DISCRDHNAWT FUNCTION 
COEFFICIENTS 



STAND^IZSD 



UNSTANDARDIZED 



0 .23698 
0.32092 



0 .155 



5^ 



-0.21587 



0 . 82497 



0 . 80084 



-0 .38896 



-0 .7'^^15 



0. 45025 



4 .77215 



0 .31550 



-116 .51265 



-0 .20194 
0 .19126 



-0 .0006-3 
0 .01511 



0 .13295 



-0 .01648 



0 .00197 



0 .27081 



'0 .03865 



-0 .28957- 



. 0.02183 
-0 . 14455 

0 .0T4I2 

-0.20297 



CANONICAL CORRELATION OF DISCRI>JIKANT FUNCTION IS 0-617 
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TABLE 17 - 'EXPANDED ^OBJECTIVE PARAMETERS TURKEY SH001 ^ 
winner" AND RUNNER-UP PREDICTORS ^ 
— ' ' 4 r— 1 

FREDICTQR VARIABLES: Expanded Data Set (Without Demographic 

Data) 



ACTOAL GROUP 
MEMBERSHIP ^ 



GPI 



Turkey Shoot-' 
Winners and 

Runners-Up ' 

Third, Secbnd,** 
and First Elimi- .GPU 



nators (/Dthefs) ' 



NO. CF 

CASES 



23 



66 



HIEDICIED GROUP MEMBERSHIP 



GRQUP I 



19 
82 .6% 



16 
24.2% 



GROUP ii; 



4 

17.4% 
-4 



78: 8% 



77.5 % OF CASES WERE CORRECTLY GROUPED 



VARIABLE 



CLASSIFICATIOT FUNCTION 
' CDEFPICIENTS 



MIO 



Ml 4 



M24 



M29" 



M32 



F18 



F27 



F29 



CONSTANT 



GROUP T 



0 .02022 



0 .00224 



0 .38942 



0 .13.310 



-0 .00597 



0.42068 



5.90280 



0 .12566 



-54^3753 



0 .02543 



0 .00245 



0 . 43126 



0.17617. 



-0.00758 



ro;5122fe 



6 . 4062$ 



0 .05530 



-6 4.82 43"^ 



1)ISCRIMINANT FUNCTION 
COEFFICIENTS ' 



STANDATmT TliT) 



0.18801 
■0.19653 



-0.28432 



-0 .32723 



0.39114 



-0 .'53455 



-0 .401832 



0 .53757. 



; 



tJWSTANDARDIZED I 



-0 .00 302 
-0 .00'0r2 



-0 .02427 



-0 .0249 8 



.00094 



-0.05311 



-0 .29227 



a..o4oao' 



6 .15123 



^CANONICAL CORRELATION OF DISCRIMINANT FONCTION ^S 0 .542 



ERIC 
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TABLE 18 -'expanded OBJECTIVE PARAMETERS TURKEY SHOOT 
WINNERS, RUNNERS-UP AND THIRD ELIMINATOR (UPPER HALF) P^EX). 



PREDICTOR VARIABLES: Ex^panded Data 'Set (Without Demographic 

. Data 



ACTUAL GRODP 


NO. OF ' 


mEDICTED .QlOUP MEMBERSHIP 


MEMBERSHIP 


CASES 


GROUP I V 


GROUP II 


T.S. W-inners, 
' Runners-Up aifd GPI 
Third Eliminators ' 


46 


36 
■78.3% 

> 


10 
21.7% 


T.S. Second and ; 
First Elimina- GPII 
tors . ' 


43 


10 
2 3.1% 


33 
76.7% 



77.5 % OF. CASES WERE CORRECTLY GROUPED 



VARIABLE 



CLASSIFICAT IOW FCW CTION 
• COEFFICliNTS 



M4 



M20 



0 .02266 , 



0 .'511 



0.02134 



0 .59123 



DISCRIMIKAW T FUN CTION 
COEFFICIEirrS 



STAWDARDIZRD 



■0.22960 

'o'.'2"7909 



UffSTAWDARDIZED 



■0.00068 
0.04063 



M25 



0.0 890 



0.05773 



-0.p7943 



-0.01603 



Fl 



20.40007 



21.28377 



0.30075 



0.45352 



- ^F18 



0.21914 



0 .31037 



0.47031 



0 .04673 



F25 , 



•0.12013 7 



0.09632 



-0.25523 



-0.01216 



F29 



0.01772 



-0 . 0 4 3^2 



-0.410.56 



-0.0 3116 



F30 



-0 .56101 



0.59603 



0.30273 



0 .59262 



CONSTANT 



-126.35294 



-131.05737 



-2.39923 



CAHCJOCAL CORREIATION OF DISCRIMIRANT FUTCTION IS 0.616 
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TABLE 19 - EXPANDED OBJECTIVE PARAMETERS QUARTILE 
RANK PREDICTORS 

^ • - ■ >^ . — - 

PREDICTOR VARIABLE^: Expanded Data Set (without 

Demographic Data) 



j * ACTUAL GROUP 
1 MEMBERSHIP * 


NO. 

OF 


tRBDICTED GROUP MEMBERSHIP 


GP I 


GP II 


•GP.III 


GP IV 


iTvirXey Shoot GP I 
tvJinners & Runners . Up 


23 


14 
69.9% 


4 

- 17. 4% 


2 

8.7% 


^3 
13.0% 


iTurHey Shoot GP II. 
Third Eliminators 


23 


4 

17. 4% 


13 
56 .5% 


■ 3 
13,0% 


- 3- 
13.0^% 


Turkey Shoot •^•^•^ 
[Second Eliminators 


23 


5 

21.7% 


3 

13.0% 


*> 

12 
52.2% 


•13.0% 


kuifJcey Shoot '"^^ 
plirst Eliminators 


20' 


1 

5.0% 


0 

0.0% 


' 2 ■ 
10.0% 


17. 
85 .0% . 



62.9 % OF CASES MEBE CORRECTLY GROUPED 




CLASSIFICATION FUNCTION COEFFICIENTS 





GROUP I 


GROUP II V 


GROUP III GROUP IV 


|m9 ^ 


0.390 80 


.0.43457 • 


0.42527^ 


P .0.32973 


1 Mil 


-1.09244 


-1.19695 


-0. 84949 


' . ^ 

-i. 2 70 48 


|m22 


3.76039 


3. 83710 


3.54577 


4.03881 


I M25 


0.05883 


0.07826 


0.04751 


0.05261 


1 M32 


-0.00712 


-0.00953 


-0.00784 


-0.00910'. 


1 Fl 


23.15227 


22.54955 


2'3. 13644 


24.77510 


1 F16 


1.25992 


. 1.56965 


1.20116 


1.6,1926 


1 Fl^ 


* 0'38089 


0.42928 


0.49087 


1 — - -■ - 

0.49089 


1 F2 3 


* 0. 32194 


0.29975 


0.32001 


0^28426 


1 F^5. 


0.23929 


0.24372 


0.21273 ' 


0.24954 


I F27 


. -0.43905 


1.20003 


0.7*42 6 3^^ 


0.40397 


1 F29 


-0.04349 


-0.07499 


-0.06826 


-0.20159 


CONSTANT 


-134.68774 


-140.15710. 


-139.95496 


-147.62793 


















% ^ ~ 




1 
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TABLE 19 (CONT.) - EXPANDE6 OBJECTIVE PARAMETERS QUARTILE 

' RANK FUNCTION COEFFICIENTS : '\ . 



PREDICTOR VARIABLE SET-: Ex^iahded . Data Set (Without Deinographic 














■ <i 






• DISCRIMINANT FUNCTION COEFFIc'lENTS 




VARIABLE 


. - STANDARDIZED 


U^STANDARDIZED 






tpr>M T T 
r • X J. 


r • XXX 


r CIN • • X 


r ^IN • X X 


r v^IN • XXX 


M9 ' . 


n O Q '3 "7 o 

U . z y J / Z 


u . u D lb oy 


A C A C T 1 

U . b U 5 J 1 


A A 0 ^ A 

0 • 04^40 


A A A ^ O 0 

U .006 29 


0 .04 8 86 


Mil ' 


r\ T T O T O 


A . T O O O C- 

.0 . 7^985 


0 .. 1 J 6 5 0 , 


0.07279 


0 . 19 40 5 


0.03629 


M22 ^ 


W • ^ O *T O ^ 


U • 1 1 X J 


-0 1 7 70 9 


-0 1 1 1 1 

U • X X X X D 


— 0 1 7 9 R 


— n n 7c; 9 


M2 5 


-0.07615 

- ■ . ^ ■ — ■ . . , . 


-0.33074 


0 .24529 


0'. 00 322 


-0 .0139 8 


0.01037- 


M32 " 


0.21123 


0 .27,7 40 


0 -. 5 2 465 


0.00051. 


, 0 .00066 


-0.001'26 


Fl 


-0 437RQ 


• W (J (J ^ ^ 


-0 4 7fiR3 


-n fifiii "? 

\j • u u y V, <J 


0 .1 "? "? T«i 

\J • *X ^ ^ X ^ 


-n 7 1 Q04 


F16 


-0 22091 

(J 


-0 . 35071 ' 


0*19407" 


-0 113 32 


-0 .1799:0 


0 09955 
\j • \j -J J -.J -.J 


r X o 


-0 .31037 


0 38222 


b 35250 


-b 03084 


0 03798 


0' 0 3503 

c 


F2 3 


U • ^ D X J 4 


' n 9 0 7 17 
U • ^ U / X / 


U ^ X^ o X X 


A n 1 9 1 Q 

' U • U X ^ X i7 


n n 1 h 0(4 


-o nn 9 V * 

U . UUDZX 


, F25 ■ " 


\J • X U D D T 


All 


U • X o o u / 


— n n n R n R 


—0 n 1 7 R^ 

u • u X / 0 D 


-0 nn ^ 7 4 

U.UUDj** 


F27 


n 1 9 Q n n 


^ n 1 7 9 9 


n 77 c; 47 ' 


n n Q 9 7 4 


— 0' 1 9 4 9 . 

U • X^ D ^ 


U . 3 3 3 U / 

i ; . • 


F29. 


n 7 9 7 7 Q 


n » 1 n 9 R 


0 ' 1 0 0 7fi 

u • XU U / D 


U • VJ _J ^ ^ 




n on 7^^ R 


CONSTANT 








3.90265 


-0. 10029 


-2 . 1916 4 


■ ^ 














CANONICAL 














CORREL'. 


0.647 


0.529 


0 . 440 












p " 






% • 


'• • ~ ~ ■ — ■ ■ 
















. . * 








































V 
















i » ~ 














•* «.'.'• 
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TABLE 20 - CANDIDATE OBJECTIVE PREDICTOR VARIABLES 



DESIGNATION 

F29 
F12 
F17 , 
F04 
F06 i 
F18 
•FOl 
■>25 
130' 
M17' 
F09 
> M32 . 

M12 - 

F32 
' F08 
M13 
F22 
F2 3 
F02 
Mil 
MO 9 
F31 
MIO 

M2 5 
M16. 
Fll 
F30 
F20 
F19 
M22 
M20' 
M29 ■ 
F03 
F16 
M24' 
MO 4 
Ml 4 
M0 8- 
M31 . 



DESCRlf'TION 

HIT/MISS H-MISS SCORE HON (H* (H+M) /HON) 
*TIME TO FIRST KILL ( SEC-AVG/HEAD-ON) 
TOTAL NO. HITS HON (HITS/HON) ' 
TOTALI^-UEL USED (LBS. AVG/HEAD-ON) 
*PERCENT OFFENSIVE TIME (% AVG HD-bN) 

TOTAL TIME IN H-MIS ENV CTK -(TIME/CTK) 
' MAX QVS .(MAX/SERIES) 
■TOT..;-mME IN PUN-ENV. HON (TIME/HON) 
hit/mi^ R-MIS score HON (H* (H+M) /HON)' 
TOTAL >fJO. HITS HON (HITS/HON) 
TOTAL TIME SR LT 1500 (SEQ^AVG/CTK) 
HIT/MISS GUN SCORE (•H*TOTAL RDS/HON) 
[■IME TO. FIRST .KILL (SEC-AVG/HEAD-ON) 
S^^^^fSPREADff^N.^S^^ ^ J^N G) 
^fMlWMfS-S^m: SCORE .(H*T0TAL RDS/HON) 
,>TOTAL ROUNDS FIRED (NO". -TOTAL/HEAD-ON) 
* PERCENT TIME. IN PANG (% AVG . /CINETRACK) 
TIME TO GJJNv'ENVELSPE CTK (TIME/CTK) > 
TIME TO GUN' -EnWlOPE HON (TIME/HON) 
NO. TIMES OVER^. (TOTAL SERIES) 
TIME • TO. PANG. (^EC-AVG. /CINETRACK) ■ 
TOTAL TIME^ SR LT 1500 (SEC-AVG ./CTK) 
HIT/MISS GUN SCORE (H*TOTAL RDS/CTK) 
*AVG.- MILL ERROR SR LT 3000 (MILS^^AVG. /■* 
CINETRACK) ' ^ 

TOTAL TIME IN GUN ENV HON (TIME/HON) 
TOTAL NO. HITS CTK (HITS/CTK) 
TIME TO PANG (SEC-AVG. /CINETRACK) 
HIT/MISS RrMISS SCORE HON (H* (H+Ml/^loN) 
TOT TIME IN R-MIS ENV CTK (TIME/CTK) 
TOT TIME IN H-MIS ENV HON (TIME/HON)/ 
TIME TO GUN ENV CTK (TIME/CTK)'' . 
TOT. TIME IN R-MIS ENV CTK (TIME/CTK) 
HIT/MISS H-MISS SCORE BON .(H* (H+M) /HON) 
TOTAL FUEL USED (LBS . • AVG.yCINETK). 
TOTAL. NO. HITS CTK (HIT^/CTK) 
TOT TIME IN GUN ENV CTJ< (TIME/CTK) 
TOTAL FUEL USED (LBS. AVG/HED-ON) 
DELTA ENERGY STATE CTK ( INIT-END/CTK) 
TOTAL ROUNDS FIRED (NO- TOTAL/HEAD-ON) ' 
HIT/MISS GUN SCORE (H * TOJ'AL RpS/CTK) 



Variables used to compute GSI scores.. 
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TABLE- 21 - EXPANDED -OBJECTI'/E* PLUS'- DEr-JOGimPlfrC f Ar6\,MIi:TERS 

TURKEY SHOOT -WINNER PREDICTORS ■ ' 



PREDICTOR VARIABLES;. Expando'd' 3 Jt , Ihc j,udir.g Demoyraphic 

' • , ■ Data • ,, ' • ' ■ ' . ' \ 



ACTUAL- GROUP 
MEMBERSHIP 



NO. OF 

' CASES 



PREDICTED GROUP MEMBERSHIP 



GROUP I 



GROUPJX 



■i'urjiQv Shoot- 
Winners 



GPI 



12 



; 10 

3 • 3 b' 



Turkey- Shab-t Uon- 
Umner^ (Others) GPU 



77 



11. 7% 



88. 3^6 



87. b ■ OF CASES 'WERE CORRECTLY GROUPED 



VARIABLE 



CLASSIFICATION FUNCTION 
COEFFICIENTS 



GmUP I 



GROUP n 



DISCKINaUAST FUNCTION 
COEFFICIENTS. 

STANDARDIZED UNST^NDARDIZSI^ 



D5 



MB 



h .0.00135 
0^00331 



- 0.0001b 



0.00535 



- .0 . 09062 



0.02737 



0. 2 6212 

- - > 

0 . . ' 



M3 2 



Fll 



_F_lo__ 



■'F2 2 



F2 3 



•29 



. CONS T a: ; 'i 



0.00116 



- 0-.'002b)o 



0.98447 



f: 



0.64778 



_0_.^d124 



c 



. u^J06 2 ^ 
_ ^0_._141_:?-7 



i U.69_2_2_7 
■Q . 49 72^ 

■Ov 16497. 



0.00041 

:< . 0 0 u 7 0 

0.02181 



-0 .00142* 



-0 . 28514 
-'0 . 0^89 6 7 

^_0__.,g3p^8' 

_0_^._2_70il_ 
<^0 102-411 

-^0 . 01252. 

~0..|72 6a', 



CANONICAL CORRELATION OF DISCRIMINANT , FACTION IS 0.62 0 . 

9 
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TABLE 22 - EXPANDEQ .OBJECTIVE ' PLUS DEMOGRAHIC PARAMETERS •< 
TUFO^EY SHOOT WINNER AND • RUNNER-UP PREDICTORS ' 



1 HlEDICfTQp VARIABLES: Exl 


sanded Data Set Including Demographic 

■ . ' ■■• ■ ■ 


1 ACTUAL ^GSOfDP 
■ MEMBERSHIP 


NO. OF 

^ CASES 


HIEDICTE2) GROUP MEMBERSHIP 


GROUP I 


GROUP II 


iTurkfey Shoot . - 
Winners & ' GPI 
jRunnens Ud , : 


23 . 


19 

82.6% . 


4 

! 7. 4% 


jThird, Second &: 4 
IFirst Elimina- . GPII 
liiors (Others) 


66 • 


.12 
. 1.8.2% 


.54. e 
81, 8% 


82.0 .. %^0F CASES WERE CORRECTLY GROUPED ^' 




. .VARIABLE 


glassificat; 

COEPFI( 
haJWjyjr J. 


ION FlicTION 
-lENTS ; . 

GROUP rr : . 


DISCRIMINA] 
COEFFK 

S7AimART)T71?n 


(^T FUNCTION 
:IENTS ^ 

UNSTANDARDIZED 




n (\c\ c Ck ci 


.- 0.005,'?.7__... 


• » 35 355 
- 0.f4499 


0.00055 


1 D6 


0 D0329 


0.00079 


Y D7 • 


' 0.02,3 41 \ 


/ 0.05347 


0.4235 8 


0.01171 


M9. 


0.46 85 4' 


0.52 321 


0.22037 


0.02131 


MIO' - 


0.00171 


0.01239 


0.25887 


0 .00416 


M20 


0 .-80229 


0.91792 


0 . 30951 


0.04506. ■ 


M29 ' 


0.0 4-42 9 


0.09930 


0.2 8076 


^5 0.02144' 


tl32 • 1 


- 0.00274 


- 0.00417 


- 0,.'2 32 32 , 


- 0.00056 


Fii i 


0.64870 


0.11529 


0.61456 


0.20782 ' 


L F18 


0;34346 


0.43268 


0.34989 


0.0 347 7 


L_ F22 


0i57578 


1.09963 


0.52 312 ■ 


0.20411 


F2 7 


7.. 10 4 80 


7.60449 1 0.27220 


0.19484 


F29 


0. 1933 8 


0.Q6 6 5Q 


- 0.44603- 


- 0.03385 


F30 


0.47268 


-i. 56218 


0 .21687 




0.42454 


CC^STANT" 


-62.57329 


-73^.0 869 4 




- 4.45741- 






> 


<> 





















CANOTICAL CORRELATION OF DISCRIMINANT FUNCTION IS 0.65^ 


\ 



TABLE 2.3 - EXPANDED OBJECTIVE • PLUS JDEMOGRAPH^e. PARAMETERS 
TURKEY. SH^T WINNER, RUNNER-UP & 3R& ELIM> (UPPE^^^HAfcFj 



•raEDICTQR VARIABLES: Expanded Data^^Set Inaluding l5emographl(c 
— Dalfa 



ACTUAL GROUP 
MEa^BERSHIP 



NO. OF 

CASES 



HtEPICT^ GROUP ME34BERSHIP 



GROUP P 



GROUP II 



T . S . Winners , Runners 
Up & Third Eli- GPI 

mn p a -h r^T^Q 



46 



37 
80.4% 



9 

19.6% 



T.S. Second & 

First Eliminators GPU 



43 



7 

16.3% 



36 , 
83.7% 



82.0 % O F CASES WERE CORRECTLY GROUPED 
VARIABLE 



P 



CLAS^IFICATI Caf FO TCTION 
/C0EFFIC;iENTS 





GROUP T 


GROUP II 


STIAin)ARDT2ED 


UTS'CANDABDTZED 


nn 




0.02656 


0.27223 


-0.01065 
- 0.00057 ^ 


M4 


0.02228 


0.02106 


- 0.19296 ■ 


M20 


0.52173 


0.62508 


0.33042 


0.04811 


M25 


0.00378 


- 0.03865 


- 0.46721 


- 0.^^9 74' 


M29 


0.33286 


0.36408 


0.19048 


0.(}|454 


> Fl 


21.99968 


23.02592 


0.31727 


0.4^843 


Fll 


1.48071 • 


1.35444 


- 0.17362 


- 0.05871 


F18, 


0.07518 


0.172 84 


0.45738 


0.04545 / 


F25 


0.^1372 


0.18771 


- 0.25382 


- 0.01209 


_ F29 


0.11230 


- 0.19010 


- 0.47705 


- 0.03621/ 


Fiso 


b 0.21794 


1.52132 


0.309 84 . 


0.60653 


CONSTANT 


-139.3415^ 


-144.61646 




- 2.44321 




y 




r 








































o ; 

\£ 






V 



DISCRIMINANT FUWCTIOH 
COEFFICIENTS 



CANCWICAL CORREIATIOl OF DISCRBORANT PWCTION IS 0.642 
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TABLE 24 - ElXPANDED. OBJECTIVE PLUS , DEMOGRAPHIC PARAMETERS 
QqAR TILE RAHj^ predictors' > 

^ ■■ r-^ ■ - ■ 

PREDICTOR VARIABLES: Expanded Data Set Including 



ACTUAL GROUP 
MEMBERSHIP - 


NO. 

OF 


PREDiqTED GROUP MEMBERSHIP 


GP I 


GP II 


-GP III 


GP IV 


[Turkey Shoot GP I 
jViinners & Rvmners Up 


2 3 


16 
69;6% 


3 

-13.0% 


0 

0.0% 


4 

17.4% 


Turkey Shroot . GP II 

Third Eliminatbrs 

I • , -'V- 


23 


3 

13.0% 


14 
60.9% 


3 

13.0% 


. 3 

13.0% 


Turkey Shoot GP III 

Second Elimi<nators 

' ' - ^' 


23 


26.1%" 


2 

>^8.7% 


11 
47.8% 


4 

17.4% 


Turkey Shoot " 
First Eliminators 


20 


0 

0.0% 


2 

10.0% 


2 

10.0% 


16 
^.0% 



CASES WERE CORRECTLY GROUPED 



VARIABLE 



CLASSIFICATION FUNCTION COEFFICIENTS 





GROUP I 


GROUP II 


GROUP III 


GROUP IV 




- 0.00053 


0.00084 


0.00077 , 


0.00 314 




1 - 0. 00063 


- 0.00389 


~V 

0.00306 


- 0.00544 


M9 


1 • .0.46314 


0. -48751 


0.48939 


•.0.40282 


MIO ' 


1 ■ 0.00324 


0.01468 


^ 0.00304 


O.W122 


1 Mil 


1 0.5«092 


.0.61789 


0.9 32 29 


0.51453 


1 M12 


1 0.65554 


0.70357 


0.67359 


0.68348 


1 M2 2 


1 1.22072 


1.11891 


0.85281 


1.35006 


M25 


0.342.58 


0.-38840 


0.34136 


6.35504 


1 F16 


1.05453 


1.29887 


1.02676 


1.37675 


1 F18 


0.25768 


0.30279 


0.35183 


0.37553 


1 F22 


0.60374 


0.65370 


0.82041 


0.42426 


1 F23 


0.008308 ; 


0.05169 


0.07938 


0.02629 


f^~-^^29 ' 


0.43946 


0.40639 


0.39330 


0.28632 


1 COKl^TANT 


43753 


-101.05052 


-97.88803 


-94.15775 


















V 




1 











€9 
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TABJLE 2 4 (CONT.) 



•P^|^)ICt6r VARIABLE, SET: Expanded Data Set Including 

f Demographic, Data 

. '. fi? • ' ° ' ■ ■ 


VARIABLE 


S 

FCN. I. 


DISCRIM 
TANDARDIZ 
FCN. II 


INANT FUNC 
ED 

FCN. ■ ill 


TION COEF 
UN 

FCN., I 


'FICIENTS 
STANDARD I 
FGN. II 


ZED . / ■ - 
FCN,. Ill 


D5 , 


0.&772 4 


'0.0 6275 


-0. 0 8149 


: 0.00106 


o.oooia 


/-O. 00013 






-0.02166 


Q. 59 489 


-0-00130 


-0-00005 


0.00127 


M9 


-0 .22317 


0.11192 


-0.38708 


-0.0215 8 


0.01082 


-.0.03743^ 


MIO 


0.33817 


0. 329 89 


-0.16183 


0.00543 


-0.. 00530 


-0.00260 




-0.16161 


0.78326 


-0.39563 


-0.0-497 


0.20825 


-0V10519 


M12 


0.13758 


-0.16675 


-0.5 812 8 


0.00^648 


-0.00785 


-0.02737' 




0.1691^ 


-0.50859 


• 0.43156 


0;06602 


(-0.1^^4 8 


a. 16 842 


- M25 


0.0 642 8 


-0.42895 


-0.6 32 71 


o 0.0'0272 


„-0.01813^ 


^0. 02674 


FJ,6 


Q. 19039 


-0^.2 4227 


-0.1333§ 


0.09-766 


-0.12428 


-0.06842 


F18 ' 


■ d'. 31104, 


0. 35562" 


-0. 14287" 


0.03091 


0.03533 


-0. 01420 


F22 


-0 .18372 


0.32385 


— — — » 

-0. 31009 


-0.0 716 8 


0.12636 


-0.12099 


F23 


-0-35169 


0.25426 


0.^2047 


-0.01705 


0.01233 


'0.a0584 


,: ,F29 


-n.5q7qn 


-0. 3 5839 


-0.04.i3H 




-O.nOTQfi 


^n^m 2 0,2 


CONSTANT 


-- 


V 


— 


-0.23871 


-0.59286 


-7.7299 6 
















CANONICA], 














dORREL. 


0.679 


0.518 


0.450 










— ■ 
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functions of the analyses have greatly increased over 
cUia;Logous functions in the previous analysis, indicatir(a 
increased capability to discriminate between groups, m 

. incxeased ditecrimii;x:ant , capability is at the cost oi in- 
creaseci ^c^omp laxity in the number of varxaBles j^equired and 
the complexity of calculations. The classification fijbc- 
tiong provide optimal predictors for the objective data, 
analyses ^n this study and include the best predictor varia- 
bles, consistent with , the V^ilJ^s ' iLan^ selection 
criteria.. The two-group- analyses (Tables 16, .17> and. 18) 

^provide correct classification in4:o the top groups ojtf the 
order Cf ^80, percent ; however, a fairly laxge number of non- 
Group I members: are still being placed in these groups,' 
.'^ ■"• ^ , ^- ■; ■■ , ; ' ^-^ 

The Fourth Discriminant^Ahalysis - Statistical Der^iviation 
of a Turkiey Shoot . PI aG^meot Predictor aJsing Expanded 
Objective Parameters Plus Demographic Parameters as Candi- 
date Variables^ ~~. . T ' ' ' 

^ ' .1 ■ ■ 

/ The results of the fourth dij^ecriminant analysis are pre 
sen£ed in'Table,s 21, 22, 23, and 24, _ The analysis' uses^ as 
candidate predicto};. variables all ,of the. pr^di^tor variables 
reflected in th§- third ajialysis plus seven v candidate 'demo- 
graphic variables, ^These specific demographic candidate . 
variables-^, Table 25/ werer-av&ilable, for, all students; thus, 
no sample size reduction was required, 

X TABLE 25 7 CANDIDATE DEMOGRAPH^fC VARI^LES 

DESiGi^TioN Description y 

D4 ' TOTAi. PILOT FLIGHT jpME flpRS) • 

■ D5 TOTAL PILOT FIGHTER TII4E ' r ■ - 

DB TOTAt PILOT F-4 TIM^ (k/C & IP HOURS) 

D7 ^ ^ \^OJAL SORfflES LAST SIX MONTHS 

DIO TpTAL EFM/ACM SORTIES . ' i ^ 

' Dll *. • /^FM/ACM /SORTIES LAST SIX MONTHS - 

.013^ ■ ( time SINCE L^T BFM/ ACM (WEEKS) • ^ 



'.■ • r ■ * ; 

The objdctiare of the fourth analysis was to invest^!^ 
gate, tiie possibility of reduction of ' mis.-classif icatior 
of cases into Group I while maintaining i^ompa'rable . predic- 
t^ion ra^tes. Comparison of the prediction results for' the 
fourth analysis with thQse of the ,tliird indi^te that th% " 
fourth analysis predictions, were as/good ,or better than the 
third analysis. Mis-cla:ssif ication into Group I was redu(2ed^ 
iny three' of the four classifications, and correct classi- 
fication intd Group I was improved slightly in two of the 
four , classifications. Evidence ^f tljis improved ■ discrimi- V 
nation is provided by improvements (increases) in the 
canonical correlations of the discriminant fiinc^ions.^ 

In the first c^ssif ication scheme (Group I - Turkey 
Shoot Winners, Groiip ir - Other) the number of prediptor 
variables required to maintain a constant correct classifi- 
cation rat^ was reduced from 11 to 10 by inclusion of demo- 
graphic data. ' , ' . - 

Discussion of Third and Fourth Analyses ^ ^ ' 

> * ' . . • ■ . 

the third analysis, over 80 predictor variables were 
availaM.ei for consideration' as csmdidates' for the-' analyses . 
Th^e v'^iab'les. jwere calculkted using th^ master data-base 
which Vought constructed during the first part of this y 
study., i These ^data include the expan<!led list of 12 .variables 
which were* required- by. the contract to be analyzed. An 
•initial screening of the complete list' was necessary to-/, 
reduce the number vOf variables to an . accfi^ptable si: 
TJiis screening was accomplirshed by correlating ally 
variables with turkey shpot rank and .theif selecting the 
40 variables from the list with" the greatest correlation. 



co^f icaents.r . The 4 0 candidate variolas are oresented 'in 



Table' 26 by rank as determined by the .absolutfe values of 
the correlation coefficient (R) . . Variable designations are 
co^ed so as to indicate the class day oh which each is 
^ collected. For example, F29 indicate^ that the variable 
value is' collected on Friday (the "F" prefix indicates 
Friday) , whereas M30 is a variable 'for which data are. 
collected on Monday; Table 27 shows those objective varia- 
bles which- were selected by DISCRIM as the best turkey shoot 
.rank predictors. In thi^ table, the predictor variables are 
separated by day of data collection. The discriminant 
classification -schemes byVhicj? each are used is also indi- 
cated. UsB 'pf this expanded list of candidate variables 
appears to have generally improved ' the winner prediction . ;, 
capabili'ty. ' . 

In the' fourth analysis, a selected set of -^even demo- 
graphic variables were introduced. These w^re" selected 
mainly on thefcasis.of sample completeness, as it was not 
desired t(^ reduce, the* sample" size by excluding cases where 
incomplete data sets occurred. Non^quantitative data were /' . 
also excluded. All objective variables , selected in the 
third.. ana<lysis were retained, but objective data cQhsidered , 
in the third analysis but not selected were excluded. 
Table 28 defines^the variables considered in the fourth 
analysis. ^Npte that ''D^ii^the^^aria^e prefix used to de- 
signate the demographic vaWables considered.;: As can be 
seen from the table, inclusion of the demographic data caused 
■several Monday '("M" prefix) variables to be excluded. Also, 
as a resul-t of the. additi^on of demographic data in the analy- 
sis,*, certai-n other variable selection changes occi>rredJ ' 



TABLE 26 - 




ANDIDATE OBJECTIVE PREDICTORS RANKEe(,3:A " 
dOFfRELATION COEFFICIENT WITH ACTUAL TURKEY 
SHOOT PLACEMENT 



^ V DEFINITION ^ 

F29 HIT/MISS H^MISS SCORE HON (H* (H+M) /HON) 
F12 *TIME TO FIRST KILL (SEC-AVG/HE/tb-ON) 
F17 TOTAL NO HITS 'HON (HITS /HON) 

TOTAL FUEL USED- (LBS. AVG/HEaB-ON) 
*PERCENT,6FraNSJ[VE TIME (%,AVG HD-ON) 
TOTAL. Tira: IN H-MIS ENV CTK (TIME/CTK) 
MAX G'S (mX/SERIES) 

TOT. TIME IN GUN ENV HON (ilME/HON) 
HIT/MISS R"MI^ score!: HON ,(H* (H+M) ./HON) 
TOTAL, NO. *HIT^ HON \HITS/HON) " ' _ 
TOTAL' '^I^ME SR LT 1500 (SEC-AVQ/CTK) 
HIT/MISS GUKSCORE (H*TOTAL RDS/HON) ^ 
M12 *TIME^TO FIRST^iXiL (SEC-AVG/HEAD-ON) . 
F2 7* G SPREAD HON (MAX^G - MIN G)'' 
F32 HIT/MISS GUNfSCORE (H*TOTAL RDS/HON) 

iTOTAL ROUNDS FIRED' (NO. TOTAL/HE Ad-ON) 
*PERCENT TIME. IN PANG (% AVG .-/CI NET RACK) 
TIM^) TO GUN ENVELOPE CTK (;^IME/CTK) 
TItffi! TO GUk ENVELOPE HON (TIME/HON) 
NO. TIMES PVER G (^OTAL ^ERIES) 
TIME.-TO PANG (^C-AVp./CINETRACK) 
TOTAL TIME Sa. LT iWo (SEG-AVG . /CTK) 
HIT/MISS GUN^SqORE (H*TOTMi RDS/CTK) 
*AyG. MILL ERROR 3'R LT 300Q (MILS-AVG./ 
CINETRACK") 

TOTAL TIME IN 'GUN' ENV HON (TIME /HON) . 
TOTAL NO. HITS CTK (HITS/CTK) 
TIME ' TO PANG (SEC-AVG. /CINETRACK) 
HIT/MISS R-MXSS^ SCORE 'HON (H* tH+M)" /HON) 
^TOT TIME MN 'R-KIS ENV CTIC (TIME/CTK) ' ' 
'tot time IN^ H-jMIS ENV HON (TIME/HON) . 
TIWE TO GUlil ENV, CTK (TIME/CTK) ' . 

T0$. TIME IN R-MIS ENV CTKy^ME/CTK} 
HlT/MISS H-MISS 'sCORE HON. ^H* (H+M) /HON) 
TOTAL-FUElAUSEb (LBS . AVG. /tiNETK) 
TO^L NO.- m:TS CTK (HITS/CTK) 
TOT TIME IN GUN ENV CTK (TIME/CTK): ; ^ ^ 
■j'OTAL "fuel USED (LBS. AVG/HEa-ON) " — 
DELTA "energy STATE CTK (INI^-END/CTK) , 
TOTAL ROUNDS FIRED (NO. TOTAL/HE ADfON ) • 
HIT/MISS GUN SCORE (H * TOTAL kDS/CTKj 



to compute GSI scores. 
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TABLE 27 -''sELEG'fED. OBJECTIVE DISCRIm|jANT^ARIABLES 



/ 



W b 
20 

H'W 



X 



If 



s 

w 

CD 

> 



X 
X 



X 
X 




\ 

H W 

ft; 0 
w 



X 



X 



X 



ft 
Pi D 

0 a; 



X' 



X 
'X 
X 



1 /■ ' 



VARIABLE -DEFINITION^' 



• TOTAL fUEL USED' (LBS./AVG./HEAD-ON) 

'total rounds fired" (no. total/head-on) ' 
■ total time 'sr' lt 1500 (sec-avc/oinetrack) .. 
avg;. errok sr,lt 30oo (mils-avgVcinetrack) 
time to.pangmsec-avg./cinetraek) ' .' ^ 
time to ri'rst kill' (sec-avg./heat)-on) 
. delta energy state - cinetra,ck (int. - end/ctk) 
total no. hits - cinetrack .(hits/ctk) 
total time in r-mis envelope - ctk (time/ctk) 

TIME TO GUN. ENVELOPE - .CINETRACIk (TIME/CTK) 
TOTAL TIME IN GUN ENVELOPE '-• CTK (TIME/CTK) 
. TOTAl 'TIME IN GUN ENVELOPE - HEAD-ON (TIME/H.ON) 
HIT/MISS HEAT -*MIS. SCORE - H.'ON' (H*(H+M)/tr.ON) 
-M'/MiSS GUN. SCORE (H*TOTAL RDS/H.ON)' 



-(1 



MAX G'S (MAX/SERIES)' ' 
TIME TO PANG (SEC-AVG. /CINETRACK) < ' 
TOTAL NO. HI.TS CTK (HITS/CTK) 
•TOTAL TIME IN H-MI^. ENV. CTK ''(TIME/CTK) 
^IME TO GUN ENVELOPE CTK (TIME/CTK) 
/ TIME TO GUN ENVELOP! H.ON (TIME,/H.0N) " 

/ total. time.jRj GUN.toot;e..on (time/h.on) 

• G-SEI^AD H.0|,(MWGfiN -^)^; ,^ 
HiT'^MISS- H-MIS SCSRi^H.b^'"# (,lftM)/H. 
,klT/MISS R-MIS SCORE. Ii^GN^(fe.iM^:/H^N) 



X 

X 
X'' 
X 



X 



X' 



X 



X 

X. 



X 

X. 
X 



X 

X 
X 

X 
X 

xJ 

X 



1 




TABLE 2 8 



- OBJECTIVE AND DEMOGRAPHIC DAl-A VARIABLES 
'vipURKEY SHOOT, PLACEMENT PREDICTORS 



VAR. 

DESIG 



D4 
D5 
D6 . 
P7 
■ DIO 
Dll 
D13 
M4 
M8 
M9' 
MIO 

Mil 
M12 
Ml 4 
M16 
M20' 
M22 
M2 4 
M25 

M29 

M32 



04 



y 




VARIABLE DEFINITIONS 



J 



TOTAL PILOT FLIGHT. TIME' (HOURS) 
TOTAL PILOT FIGHTER TIME- (HOURS)' 
TOTAL PILOT F-4 TIME (A/C & IP HOURS) 
TOTAL SORTIES LAST SIX MONTHS 
TOTAL BFM/ACM SORTIES 
BFM/ACM SORTIES LAST SIX MONTHS 
TIME SINCE LAST BFM/ACM (WEEKS) 



Fl 
Fll 
F16 
F18 
F22 
F2 3 
^F25 
F2^ 

P29 



X 
X 

IV 



OTAL FUEL USEp (LBS, AVG . /HEAD-ON) 
'OTAL ROUNDS FIRED (NO. TOTAL/HEAD-ON) 
TOTAL TIME SR. LT. 1500 FT . (SEC . AVG . CTK) . 
AVG. MIlVeRROR sr. LT.* 300Q FT.. (JVI^LS- 
AVG./CINEfrR^CK) \ / ' \ 
TIME. TO PANG (SEC. AVG. /CINE TRACK) 
TIME 'TO FIRST KILL (SEC-AVG/HEAD-ON) 
DELTA ENERGY STA-J^E - CTK /( INT ^-END/CTK) 
TOTAL NO. 'HITS -'^CINETRACK (HIT5/CTK) 
TOTAL TIME ' — 



X. 

X 

X 

X 



X 
X 



X. 
X 



X 
X 
X 
X 



r-msl env.'-ctk (time/ctk) 
;nvelope«ctk. (time^^cj^i^^ 



CTK (T^ME/CTK) 
- IIEAD-ON 



SCPRE > H-ON 
(j^20^L 



ROS/H-ON) 



TIME ^ TO GUN 
TOTAL TIME ITJ GUN ENV. 
TOTAL TIME KN GUN ENV- 
(TIME/H-ON) 
HIT/MiSS HEAT MIS. 
(H*(H+M)/H-ON) 
HIT/MISS GUN SCORE 
MAX G'S (MAX/SERIES) 
TIME TO PANG (SEC . -AVG./ciNETRAtK) 
TOTAL NO. -HITS, CINETRACK (HITS/CTK) . 
TOTAL TIME IN' H-MIS . ENV. CTK . (TIME/CTK) 
TIME TO GUN ENVELOPE HON.. (TIME/H!0N.^ 
TIME TO GUN ENVELOPE HON. '(TIME /HON .) ^ 
T.OTAL- time' IN ,GUN ENV. HON, (TIME/HON?), 
G-SPREAD HEAD-ON (MAX. G-MIN "5 OVER 
SERIES-) ^ ^' ^ ' 

HIT/MISS H-MIS SCORE' HON (H^H+,M) /HON) * 
HIT/MISS; R-MIS SCORE HON IH* "(H+M) /HON) 



EKLC 



.4 



1 



Comparison ^of Prediction Results 



liable; 29 summarizes the predictive capabilities of the* 
major predictor models presented, /fhe table also includes 
approximately 95 percent confidence limits on the -prediction'* 
ratios . Note that the confidence limits dre approximate 
and use th^ normal approximation to the binomial." This re- 
quires, a relatively, large sample size. For predictions of 
the winner, (the last row of. the tab^e) / sample size is nine 
or 12. . • 



Tests of the Predictor Models 



« 



"Given,, the predictor^_n^delS developed using disci-imi- ' 
nant analysis/ it is necess'ary to test these models using 
data collected, outside- the ^experimental- data set. The-/' 
purpose of these' tests is to determine if the predictabi- ■ 
lity of the developed models is retained using predictor 
variable data-Jiot u^ed in the calbulation of th^ parame'ters 
oy in the selection e>£. the predictor variables. in the' ' 
analysis performed/ there is- evidenc'e .that the pa-rameters, ' 
selected are very sensitive to the particular'data' set .used 
in their estimation^and^ to 'the definition of the dj^drimi- ; 
• • nant groujSs. The values of . ^e, parame|;er estimates- are' a^so . 
probably quite sensitive to the data set-losld. " ' 

-Avery limite* test analysis^ us i^^/^'ata, obtained prior 
•-^ to this s-tudy^as been conducted pif the predictor models 
ddveloped f rom the firs t^d sec oV analysis defin^4 pre- 
vioisifcly. -In "the first analysis,- Monday arid Frid; 




Ostle and Mensing.^4 .gtatisiics in" research . (3rd'ed,). 
J Ames: Iowa State University Press, 1975, 100-101. 
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GSI scij^^s-^^We^e the predictor variables. in the second 
^analy^svVthe prisdictoc variables were selected Monday 
an(i?^'riday..GSl cqinponent^variables . No additional analysis 
-has been" copaufcted on outside dal^a for the third and fourth 
.analyses becabyfe it has been determined that the required 
data ^re not available and/or ncbt available to the extent, 
necessary, for reduction to the kq^ter data base form. 

Two other difficulties were also encountered ^n ac- - 
, quiring^ prior datk for model testing. First, adjustments 
had been made -in the weighting factors used in calculating 
GSI. Thesis adjustments were nbt ciocumented, and thus, a 
consistent set of historic GSI scores is. not 'readily avail- 
able. The second difficulty encountered p^tains to the 
pridr record keeping 'procedures on GSI component variables. 
The automated\GS I component variable J^porting. forms were' 
implemented be^ginning with TAG ACES I Class #7815. Thus, ' 
nominally, GSI-- component variable averages were not con- 
sistently Recorded in -a usable ^^irm prior .to Class #7815. 
Further, Class #7816 had' missing (^ata -|pr Monday GSI c©m- • 
ponent variables. . For; Classes #7832 #78-33,, tWo classes 

held 4ffcer the study samiJle it was determined that turkey- 
shoot compilations" were cond.upted In an irregular manner; 
t|hat'is, vcertain .cpmp'etiti^i^^ were Mbetntinated when two con- 
tes tarits were eiiminateVplimulta IV . by air-to^iir * 

'Collision. Tjiis pract/ci j^feempted evaluation 'of *^rkey - 
shoot results,:„using, the". miethod used previously in defining 
r.anks. Thti^, ciassif ication of re^ultsVcould, not be*''deter- 
mined using definitions defined for thfe discriminant pre-» 
die tor .model. ■ t. ' ^' j ' l 

• The results of these'" data' restriqtions- limit the .,' ' 
analysis to four classes .-(.7815, 781.7 ,'r7'8l8^ and 7819), 
totaling 30^tudehts. It. is also ^es^tricted tc?. predictors 
using G^.I and GSI component variables. This, of course, ' 



precludes evalua^tion At this time of . the best predictor - 
models; that is ^ those using the expanded data se^t and 
deirfi^graphic data. Recommendations are made at the' con- 
elusion of this report that will alleviate these restric- 
tions. 



Evaluation of Predictor Modelg Using Monday and, Friday 
GSI Scores ^ ^ . 



The first comparison conducted was ^for grouM.ngs 
where the top group was, defined to^ be winn^r^' only^ and the- 
second/group contained all others. Figure 12 ^graphically 
shows the classification of the data from the original 
' (experimentV datiV The graph shows^ond^y GSI (-MCSI) 
plotted verisus Friday GSI (FGSI) , The line shown is o^B- . 
tained by setting^the Group ^ classification function 
equal to the Clas^ II classification and solving for FGSI 
as a function of MGSI, All -pogLnts above the line are 
placed in . Group I (winners) while all points falling below 
t^l^ line fall in Gr6up II (others) , .Figure 13 shows a ' 
similar plot of the test* data u^ing the saune discriminant 
function developed frofli^the experimental data, A statis- 
tical .test of the rvull hypothesis that the proportion of 
correct classifications (P„) « using experiment data is equal 
'to the proportibn pf correct 'classifications (Pm) using the 
test data was condijcted^- i,e, ^ Hq : = P^-^ver'sus H^^ 



P y P^, The null hypothesis is accepted at the 95 ^er- 



. * ;;;Gent level.-; — . ' . ^ . i 

Similar plots are presented: (Figures?' 14 ^ ^15 .16 ^ and 
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17) showing alassiiEications the experimental and test 

OS 1 1 e^ ^d Merfs in g , ' « Statigtidg; diy^research , ( 3-rd ,e d . 
AInea^^< Ibwa St^te .U9i,versity Press ^ 19 75, 135-13*?,- 
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Figure- 14 . c lass i f iia^'ixai^ diagram"-^ -^experiment data 
/(Group- 1- ^ *?;Jinaier^ and runners-up; Group 
, . IT .= oth'ers):*. ■ "* , ' • . • ; 
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Figure<,17. Classification diagram - test data. 
■ 0 (Group i = winners , runners-up,^ and 
J/ third^eiimiriators ; Group^II = second 
* I and first eliminators) . 
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•aata /fj^the'-.btljer .tWo.-gxoup ''def ini^'iohs defined in thp,. . *' 
first' bnal,ys is / 'i.e, , Group I = winners and gunners- up ' 
versus Group II = others, anf Group I = winners/ runners- - 
up and third eliminators versus Group II = second and 
first eliminators). Similar tests of hypotheses were also . 
conducted and accepted,, i.e., no* difference in predicti'oi^jp 
rat;es between the experimental .and. test data Tjjex.i^ dei^etete'a"^- i 



Evaluation of Predictor Models '-.-Using Monday and Frid ay 
GSI Component Variables \ " 

The second set pf compat^ons were made using the pre- 
^ dictor models developed from th^segend discriminant analy- ' 
SIS. The number of predictor variables selected f or . the 
models in this analysis was usually greater than two. 'For 
, this cpmparison, tabular displays were selected. Tables * 
30, 31, 32, and 33 provide the results of the test data 
classifications. For example,. Table-^^^^jO— tQP. i = Winners, 
GP* Ii -= Others) , shows the data (X3. anci X8) and- the -calcu- 
lated classification function scores FCN I and Class 
_ FCN II) used to group the cases (actual gxoup "membership .is. • 
also provided to determine correctness^ of the predictions). " 
JiS noted, previously,^ a case is classified into the group 
with, the greater classification function' score. For exam-^.,^ 
ple.^ consider the first case (X3 = 72 and X8 = 9 8) . "The / 
function I score Is 49.4, and the function II scoife is^^5\7. 
since 49.7 is greater than 49.4, the first case is . correctly 
predicted to belong to Group II,/ i.e., others or nqri-winners . 
Of the 30 predictions shown in the-tabXe, 21 or 70. percent 
-were correct. This compares to an estimated correct phre- " 
dictidn rate of^ about 61 percent for. the experimental data. 
Test;ing.the null hypothesis t^at the. correct prediction ' " 
rates of the experiment and test samuie are^ ^imalt a test 
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■/'5,0' 


28. 


: 142 


'80 


30.12 


31.31 


2 


Jo 


■ ■ •I.O-.' • 


' -1 / 


■ ''33' ■ 


50 


85.. '. 


26 


24,77; 


24.91 


2 


■ ■ No 


: ' 5.'5 ' 


• 2 . 


/ ,'8 


37: 


^7^7 


31 


13.27 


12.14, . 


. 1 


'•No • \' 


•.-No. -PredictiQns -30 
. No. Cotrect Predictions' 


= 21 ■ 




1) 








1. 





jEr|c . 



: r- It 




"■ .1: 

.'.Hi ;• 



■;.o 

•.0. 



I 



■ ;* V 

• ) 
- r 

/ 

I- 

.'* r 

O.J. 



4tO 

3.0 

.1.0' 
i5-..5 

3'.0' 

2.0 
1.0 
5.5 



Predict: 
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'1 V ■. ' 



* 




! I ■ 

\ : 




1'G2..783 
■75 . 74,2 '1 
1^5; 8i7 ' 
145.076 
112.264 
98.650 

No.' Correct >1 



,..c:.; 



■ / 



^ • ^ TABLE 33'- QUARTILE: PLACEMENTS (GP I = WINNERS AND -RUNNERS -UP; GROUP. II = THIRD - 
, 'ELIMINATQ-RS; GP. Ill = SECOND ELIMINATORS; GP. IV = FIRST ELIMINATORS) , 



10 



TUEEY ACTUAL GP. 
SHOOT ■ MEMBER- ' 



^ •; , .: > ■ ■ ACTUAL GP.' 

CF II CF n'' CF 13 CF.R MEMBER-" ' 



CLAbo KANK 




15 7.5 ^ 




7.6 • 


4 ' 


A A 


; 2 


2.0 • 

1 "A ■ 

1.0 • 

i • ■ . 


1 


T A 

J.O 

5.;5 . 


> 2 

3 

■ 3 


17 ■ '.2.0 




/I A ' 

, • . 4,0 


2 


3.0 


2^ 


■ ' 1 A 

. 1.0 


■ 1 


1.5 


4 


. ■ ■ 5,5 


' 3' 


1 r r 

> 5,5 
• 7'.5 . 


3 ' 
4 


18 2.0 


i 


,7.5 


A 


3,5 


3 


..4;0. 
I'J,. 0 


2 


1.0 


1 

1 . 


5,5 


• 3 


19 7.5 


■ , 4 


3.0' 


, I 


5.5> ■ 


3 


7.\. 


. 4 




. '1. 




1 


. 3, ' 



CORRECT 



31 

^26 

A 29 
V4.2 

.42 
■24 
■,44' 
'22 
'^40 
, 33, 
18 
36 
33 
.26 
•150 
2i' 
60 
29 

■ 33 
83 

■ v3.1 
37 

20 
25 
16 
18 
, 19 
50 
33 
8 



•72 27 




75 ■■3 
78 2 
78 12. 
■74 14 
75 42 
:79 ]9. 
63 26' 
66 43 
'-72 49 
68 12 
7^19 
38. 25 
75 '1$ 
65 30 
7a 41 
55 19 
63 42 
■64 12 
72 10 
■69 28 
75 16 
64-. 15 
72 >1 
42151 
.65 10 

74 80 

75 26 
71 31 



^98 
101 

72' 

80 

75 
.120 

72 

.57 

116. 

131' 

106 

'"95 

104.' 

100 

127 

• 80 

170 

161, 

' 89 

106, 

»78 

107' 

101. 

■103 . 

106' 

180 

119 ■ 

91 

91 

^3 



52.600, 

63..695 ; 

53.636 

59.912 

59.223 

56.159 

.55-.863 

56.091 

44.27(1 

48.851 

51;8S5' 

47.707 

60.974 

, 8.952, 

7.6.123 

40^533 

7.1. 8li) 

36.9;92 

"40.797' 

50.432' 

50.322 

^0.573 

5.4,972 

41.923 

51.334 

2'2.889 

43.874' 

57.945 

55.967 

47.617- 



52.616 
•63.573 
^4'.127 
'60.793 
60.'107- 
55.392 
57.072 
56.531 
•44.261 
48.411 
51.3^9 
47.8"52 
60.981 
8.320 
'80.945 
40-, 367 
72,001 
35.540 
41,073 
52.452 
50.57J 
50.671 
54.393 
41.406 
50,-555' 
21,'39.5' 
42.773' 
59.346/ 
56.22^ 
96. '705 



52.250- 
62)986 
'52.994 
59.522 
58.755 

■ 55.719 
..55r3i^8' 
55 .0731 
44.504 
48.940 
51.310 
47,563 
:60.525 

■ 9.504 
78.589 
40.0^0. 
72.403 
37.487 

'40.711 
51.553 
49.837 
'50.506 
54.293 
41.696 
50.692' 
23J11 
,577" 
57.962' 
55.5.30 
46.740 




i-2;-420 
5"8.992 
57:634 
.55.282 
4.718 
4.326 
44.472 
49.000 
5:. 295 
47.182 
59.653 
.11.255 
■?:".52G 
4C.236, 

3:. 228 
4C.861 
5C.467 
49..266 

'5C.162 
5; 850 
41,822, 
5C,614 
26,290 

•43.770 
57.312 
54. 855' 
46.861 



A, 
■ 1 
2 

' i 

;2 

1 
' 2 
• 2 

■3 

4 

'■2 
2 

4-' 
2. 
1 
3 

f 

.4' 
2 • 
, 2 
2' 
2' 
1 
1 
1 

.r 
i 
2 
1 



No ■ 

No 

Yes" 

'No. ' 
No ■ 
■No . 

No 

NO' .i. 
No-' . ' 
NO 
io 
No' 
No • 
No" ' 
No ^ 
No 

m 

No ' 
Yes' 
Yes 
No' 
No 
No 
No 
lio ^ 
No 
•No" 
No 
N0~ 



ERIC 



^ 0-766 was -alcula-ec i-^ .tk^s is less ^ 
' ^' ' " ^•§4, the -lypot-.esi, _-not be r spec ted. 

... is-.:-,3 9f the. gro-u; pred^cricn -- abili es. of 

rhi --P.,.- t-r.ree .di^crimi,- an-t F--exii._-, -.06-^ ,,ere ' 

als. to predictions, jr.,a(ae with z'&s a .and 

are -o- _ii Tabies ,31 , : :. and 33". Te.,.. ..e' hypo- 

ches. : . ^ ^ _ty of the • pre dictions b-rweer e xperi- 
mer : .at . . nc ^he test data .-re ai-sc c^r-^ec ^c. For 
the :;. sdic model where Gr , up I = U^pe- i. ar ; Group ' 

II ' the null hype thesis wa5\-r- je -^d. 

How.i-.,,r, tne othe^'?^^^ predictors , :.a^ hypcthesis 

wa5 reje -c -he 95 , per cen _ level. --.^ ca. e where 

Gri. ID _ .^-ners and Ruhners-Up, and Grdur ;i ^ thers, 
the -.11 -. Dthesis could not oe rejected the . 9- percent 
lev*, i ' ■ • , 

rr..^5 aves the four group predictc . .ypothesis re- *' 
jec'- d a- . e corationly acceptaiDle J^^vels . ^^-nir.ataon^pf • 
Che iaiPDle .eans and -standard ^gviations zt the p -edictor 
va: .sed .in each da^ s^t provides some evidence 

^as : w-ny me null hypothesis' was reject -d. Tat_es 34 and 
X- .- •□w -he comparisons of- sample means -r;-! standard devia- 
ti: r^ by precictor variable, data set- (e>.^=riment or test) 
^ mc ay discr_minanf group. ^Ifiherent in r --""predictor model 
^ -ac „iremen-s is that group mesibershlp pre _cti6n capability ' 
-3c. ires rhat data for which classif icati, 3 are to be made 
^.c— d be samples from the 'same distribut . :s as those used 
- -eternne-the. predictor model Itself. C .mpa;rison of 

oceans and standard deviations shows 'thi : several ^ 
^-^rrer distinct diff^ehces exist between, t.-- experiment 
•a-.- cest zata parameti^rs. An example of th--e distribu- ' 
ta.'. al di^erenc^s i§ conltained in Figure 11 where X5 , ' 
Average Mil Errot, is compared. l^cte the gr Tkt distri- - - 
ou-icnal differences, between Groups 'li, m -and -LV 



< 



* TA=- 34 - 



OF GROUT* MEA2JE 



I 

ZRIMINANT 



oat; 



LISCFJEMINAiN': 



RC'JP 


. SET 




X3 


7-5 




ROUP I - Wir -e — 
an Riinne r s - ^ - 


Experiment- 
Test 


30.4 
42.5 


70.6, 
72^ 


: ^1 


1-24 . ■ 
103.^ 


7ROUP Il'-^'^-:Lr r- 


■ Experiment 
Tes- 


43.2 
3 3. .4 


70. 2- 
■ 70.7 


- - 9 , 


j/09 . 
A02 . 


lilROUP III - Sec uz:^ 
gijLininators 


Experiment 
Test ^ 


38.5 
40.1 


.^8.4 
|4.8 


44.3 


134 
182 


JROUP IV - Fir 
-lamina tors 


Experiment 
Test 


' 3'3 . 3 
26.3 


6 4."8' 
6 8.6 


. . 6 
__.9 


137 

109 .'. 



TABLE 35 - a ^■-^ARISON OF GROUP ' STANDARD DEVIAT:;0N^ 



DISCRIMINANT 
3 ROUP . 



DATA 
SET 



DISCRIMINAI^^T VARIABLE^ 
XI X3 .<5 Xfe 



3R0UP I - Winners 
and Runner S7UP 

GROUP .II. - .Third 
E 1 imin a t or ' 

GROUP III Secon^d 
.Eliminators 

GROUP JV - First 
Eliminators 



Experiment 10.4 

yest ' ^ , 8. 72 

ExDeriment 22 . 

Test . ' 22.8^ 



4.84 22.3 • 30. 3 
5.61 21.8" 30.1 

6. "75 45.^1 41. C- 
4.11 15.9 21*2, 



Experiment . 21.1 8.29 25 
Test 45. 7f. 16.0 44* 



17 31.^ 



Experiment 
Test:'' 



13.6 10.0 113j 31.5 
5.12 • ^.(IQ 8.38 25.5 



by data set. While the Group.^I distrih .-ions' match, quit4 
well, the others change shape radabfe^l.^ ' , 



) ■ 




J 




I ♦ 



'104 

95 



80- 




, .80 

■ X 5, "AVERAGE MIL ERRCvR ,(ytON) 

, Figure 18 A Comparison BetWjiefen the 
Oistribution of 'Discriminant . VariaI51e X5 By DcLta'^'Set 
' i ' ^/ ' 96 -., ' ' • \ 



■ ' . i"' ' - '■ ^ ; . ■ ' . • ^ ■ 

- • ' . ,The ,data collected as avpart"-of chis'srudy -were ir. tv/c* 
pi;iraary forms : student pilot objective per^'.ormarice da*a in , 
the simulator and studej^it de^T^oqrapf.rc dat'^collec ted ■ rr&n: ' 
background surveys_ and questionnaires. ;r-his. section, des- - 
cribes some of che relationships chat^'wfere investigated 
oetween the student pilot's demographic/ historical back- " 
grouixd data ahd his predicted or ac'rual, perf cr.vance' m the 
air combat simulator. The major r.ata source- for comparison ■ 
was the T-AC ACES I background survey / sKown in Afcpe^-:uix B ' ■ 



)f/hich was adapted for use in the GSI study.; The questions ' 
on this survey and their responses 'were utilized to /orm 
the demographic data base. The form was completed by each, 
student in the' study s-ample (N = 89)." The questions were . • 
identified as ^demographic variables and tabulated' into ' a 
list, which is shown in Table .35, ^otal Demographic 
Variables. This list was reduced to consider for ^analysis • 
only those variables which in'cluded a' pos\tiW, of*. other , v 
th^n zero response from all of thfe-.89 subjects in th^ study. 
These are,, shown in Table 36^- a^nd .include those -'factors- which"- 
were usecJ in. 'both the correlafciofi/ analysis/ and the/ stepwise 
seiection routines. ~ " • > 

. • Several methods^ were employed to analyze 'these- data 
which Ve re cla^i-f ie,d ' into tvio groups. Group 1 -consists of 



>that body_ of datd which resulted from responses from al'l', 
89 suHjects. -G^oup 2 consists of that body of data -whiq^v 
resulted^ from' responses from differing number's '~of subjects 
in .'thJ" sample. , ; , . . 



Gtoup Data ' i . 



A correlation analysis was employed «^o estimate 'the 
fuij<:t^ional relatio^ip amqrig the Group' 1 data or total 
sample' (N = 89) of Wut^jects ' in the ' s tudy ; ' *• 

^ ■ , / • ■ ^7 ' ■ , 



TABLE 35 ■-. 



TOTAL DEMOGRAPHIC V. 

i 




VARIABLE ' 



RESPONDENTS 



Dl 
D2 

a3 . 

D4 
D5 
D6 
D7 
D8 
D9 
vDlO 
Dll 
Di2 
D13 
TD14 
D15 

Die* 

D17 
,D18' 
D19 
D20 
D21/ 
D22 
D23 
D24 
D25 
D27 
D28: 



student pilot rank 
squadron 
Wing . 




TOTAL PILOT FLIGHT TIME, HOURS . . 

•IX)TAL PILOT^P-IGHTE^l TIME,. HOURS ' 

TOTAii PILOT F-4 TIME, A/C AND IP, HOURS 

TOTAL SORTIES LAST 6 MONTHS 
-1>TYPE i\IRCRAFT CURRENT ' / 

/PRIMARY DESIGNATED OPERATIONAL CAPABILITY 

TOTAL BFM/ACM SORTIES * ^ 

, BFM/ACM 'SORTIES LAST 6 MONTHS 

feFM/ACM SORTIES LAST>IONTH - 
.TIME SJlJJCE LAST'BFM/ACM 

TYPfi A/A' MISSILES FIRED " ' - 

FWXC GRADUATE 
PREVIOUS ACES ATTEtlDED- 
LAST AGGRESSOR DACT 'FLIGHT* ' ^ - 
OTHER VISUAL^A/A*^ SIMULATORS FLOWN 
'IpOTAL COMBAT SORTIES 
TOTA^^O^ffiAT HOURS 
NOMBfeR COMBAT KILLS 

numbpr hits recorded 

NUMBER lSi\M ENCOUNTERS 

NUMBEjR HOSTILE AIRCRAFT ENGAGEMENTS 
, ^NUflBER HITS RECEIVED ' x V 

OWN TRAfNING EVALUATION 
^'ANY TRAINING^ ANOMALIE^ ' ; 



•I 



89 

■ 89 
89 
89^ 
89' 
89 
89 

■ 89 
89 

"89 
89 
89 
23 

: 1 

1^8 
89 
18 
-19 
19 
1 

■ 1 
4 . 

■ 1- 
1 

89 



1^ 



r = 

(4 

"D5 ' 
D6 
-D7 
D8 
D9 
DIO 
Dll 
DI2' 
D13 
D17 
D27 
D2 8 



TABLE' S6 




GROUP 1 DEM(^RAPHIC VA^ABLE^ 




STUDENT PILOT RANK ■ j. i 
SQUADRON , ' ' 

WING ■ ' 

T0TAL PILOT FLIGHT TIME, HOU-RS 
■TOTAL PILQT FIGHTER TIME, HOURS 
TOTAL Pi^pT F-4 TIM^^A/G AND IP, HQURS ' 
.TOTAL SORTIES^ LAST 6%^NTHS 
TYPE AIRCRAFT CURRENT 

PRIMARY DESIGNATED OPERATIONAL CAPABILITY 
. TOTAL BFM/ACM SORTIES 

BFM/ ACM SORTIES LAST 6 MONTHS ' ' . ' 

BFM/ACM ^SkBlES LAST MONTH 

TIME SINGE-^iAST BFM/ACM 

LAST AGGRESSOR DACT FLIGHT 

OWN TRAINING EVALUATION 

ANY TRAINING ANOMALIES 



89 
89 

89 , 

89 
89 

89 .. 
89 . 
&9 
89 , 
89 
89 

89 ■ 
89 

89 - 
89 



• ^- ■ ■ 

Group 1 data includes 16 .demographic. variables , each 
with a samplfe size of 89 data points. Each variable was 
examined by correlation analysis techniques to determine 
the extent of statistical relationships, with four simula- 
, *tor pe^ormance measures and one measure of predicted per- 
formance using "Expert' Opinion" . The results presented in . 
V Table 37 indicate no statistically^signif leant relationships , 
. / The tablejshows very low correiatiNon between each of the 16 

defhograpiric variables and with each of the performance mea- . 
. sures sh,own. Correlation" coefficients were also computed 
between the 16 variables and each of t:he four GSI part^core 
comjfonents for both Monday and Friday data. Again, the re- 
sulting correlation coefficients were equally as low. 
Finally, analysis was performed using those classes and 
^ siabjects with Wednesday data available/ All of the corre- 

lation matrices developed were submitted to the Flying 
■ Training Division of the Air Force Human Resourq^s Labora- 
. tory. Correlation coefficients were computed using the 
^ ' ^saine group of 16 variables' against each Wednesday part- 
score coi^onent and. the total Wednes'day GSI score. The 
. Wednesday data involved performance scores of x^nly 2 7 sub- 
jects. * The results again indicated very low correlation. 

• -Group 2 Data * • ^. a. * 

An item analysis was empl'oyed to estimate the func- 
tional relationships ai^ong the responses to Group 2 data. 
The analysis was generalized to observations due to the 
limits that are imposed on statistical inference by very 
small sample sizes. 'Sample size in this group ranged from 
^ "n=1 to N=22. Two of the Group 1 variables were ^Iso in- 

. eluded in this analysis: D-17 Last Agressor DACT Flight 
and D-2 7 Own Training Evaluation." 
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TABtE .^7 - CORRELATION ANALYSIS 



TURKEY 
SHOOT 



Dl STUDENT > 
RANK 

D2 SQUADRON 

D3 WING 

.D4 TOT. PILOT 
FLIGHT 
TIME^ HRS . 

D5 TOT.FIGHTEF 
TIME, HRS. 

D6 TOT.F-4 

TIME, HRS. 

D7 TOT. SORTIE 
LAST 6 MOS. 

D8 type ACFT 
D9 PRIMARY DOi 



FRACT. 
WINS 



-0. 0584 -0.0272 



DiO TOT.BFM/ 

ACM S0RTIE5 

Pll"'BFM/A(>l 
- SORTIES 
. LAST 6 MOS , 

D12 BFM/AGM 
SORTIES 
LAST MONTH 

DIS TIME SINCE 
LAST BFM/ 
ACM 

D17 LAST AG- 
GRESSOR . 
DACT FLT. 

027 OWN TRAIN- 
ING EVAL- 
UATION 

D28 ANY TRAIN- 
ING ANOMO- 
LIES . 



0.2551 
0.0988 
0.1835 

> 

0.2597 

0.0436 

0.2684 

0. 3218 
0..3168' 
0.1352 



-0.2 454 
-0.0881 
-0.2070 

-0.3093 

-0.1252 

-0,2 414 

-0.3689 
-0.3331 
-0vl282' 



CHF.IP 

' RANK. 



GSI 
MON 



-"0.10^1 

6 

0:,0136' 

0.1664 

0.1202 



0.0215 
-0.2400 



-0.036:). 

0.0960 

0.0864' 

0.1307 



0.1331 -0.0859 



0.0371 -0.0248 



0.0089 0.0375 



0.0215 -0.0338 



0.0595 -0.07-25 



-0.1078 0.0367 



-0.0043 

0.0109 
-0.1040 
-0.1184 

-0.0591 

0.1051 

-b.0M6 

-b.'0692 
-b.llOO 
-0.0^54. 



GSI. 
FRI 



-0.0161 010400 



-0.1099 0.0800 



0.0838 



-0.2251 



-0.0999 



-0.0641 



-0.0712 



0.0773 



0.0391 



-0.2249 



0.0901 

0.0117 
■0.0216 
■0.0959 

-0.0254 

0.0074 

0.0155 

0.0433 
•0.0271 
0..0385 

0.1537 
0.1878 
-0.0357 
-0.0540 
0.0428 



-0.2097 
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^'ariabie D-17, -i-.i-er: .-i-.;d as the .date of the sub ject ' s 
last dissiruiiar aircrai": t r-,.o-;-ar:p combat traininq. flight,, 
was included in the in ve:L^ .:-..jation ^ because of the dramatic 
effects of aggressor^tra:.air:y reported by DeLeon (1977) • 

■ Variable iden-iiies the 3 tudent- pilot ' s affective 

evaluation, of the perceived value or the training he re- 
.ceived. It was irtcluded for additional analysis^ to help 
identify outlier S'cores and to ^ssess the effect of attitu- 
dihal value's on performance. 



Q ue s t i on / An s we i Rationale . \ ^ . 

Type of^ Air-To-Air Missile Fired ; ^ 

"What A/'A-4i\issiles have you fired?" (D-14) 
AIM • 7 AIM 9 , AIM 4 _/ :ioi:?: 



^ Tweitty-two 01 t]ji^ 89 subjec 
experienced 1 a^un c h i n mis s i .1 v s 
sample size* (N ~ 22) i^:-"-' ■ " 
tioh. The su:-"oy.:; iidi i. 
fired the P^^M :\i : : V ■ - 

missiles. i'ii-^: .. 



Ai:^! 4; 

AIM 4 , 

AIM -4, 

AIM 7 

AIM 1, 

AIM 9 



■ chat they had 
- ::craf t* The . " 
■ of the popula- 

jects had actually- 
some combinatign rf these 
this group is as fbllovrs : 

0 

'0 • 

1 

1 

11 

_6 ^ . ' 
22 > " ' ' 



•^.DeLeon,?. '-i^e pe acetin-e evaluation or the pilot skill 
factor in the axr-to-air coinbat. Rand Repor';: R-2070-PR . 
January 19 77. 
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Thfs group of 2 2 subjects' were exa:uined for "their, P/er^ 
formance in cne turkey- shoot elimination. It was found/ that 
three of the' 22 suojecrs were winners of curkey. shootsv- 
Also, seven subjec::s ^30.4 percent) were found to be either 



winners or first runner s-up , and all seven had exper:j^ence 
firing both the AXM-7 and AIM-9 missile . - ' ■ 

.It was also found th^t ^ total of seven of the 22 sub-, 
jects (30.4 percent) finished i^n the last two places ' in the 
. turkey-' shoots • The CIP rankings,^ w.ere also cdmpareA f oir^this'- 
group. Of the .2 2 sub^jects, two wer^ predicted -to. 'win* the 
turkey shoot and six Wt^ire^ predicted -6p finish "in/ last -place ^ 
by their I<Ps. * « ^ . ^ y 

Fighter Weapon Instructor Course (FWIC) 

S "Are you an- FWIC graduate? .:(D-15) Yes . - . No . " . 



Of t^e 80 'subjects in the study sample, only one. of the 
students in the .TAC ACES program had. completed Fighter Wea- 
pon... Ins tructor Course (FWIC) trainijig. ^It was also, found ' 
' thit- there has been a total of ,11 FWIC. graduates O'Ut of the 
4 56 subjects completing the TAC ACES training. 

The subject had experienced , 1700 hours of rotal .flying 
time, 1500 hours of^ fighter aircraft time, and 1500 hours of 
F-4 flying time. - ^ 

A comparison . of turkey -.hoot data shov/s tha:: the sub- 
ject placed second in the ourkey shoot contest.' Bpth'his 
Monday and Friday GSI performance scores were above 700 
points. - Analysis of the Friday^GSI pair-'t scores , however 
did indicate a decline of up to 30 percent' from the Monday' ' 
GSI'^art scores. ^ ■ , ' 



■ /• 

J'-. 
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112 



"Have. you previously attended: TAG ACES I 
. ' TAG ACES 51 NONE ^ . " (D-16) 

\, . This question was included to. determine the extent of 
> the subjects experience with TAG AQES programs. Specif i- 
caO^ly, it was used to determine if any reiatipnship exists 
between pie performance, of subjects, with any or no TAG 
AGES experience, in the turkey shoot competition.' . A total 

(19.1 percent) of the ,89 sulDjects in the study respon- 
ded- that tjxej^ had previously participated in th^ TAG AGES -I ' 
or TAG ACES II training program. One of the subjects had> 
completed both .programs. For the TAG ACES I program, 11 
respondents in the sample indicated that they had completed 
the training. [ When contrasted as a group with the total 
sample of turkey shoot participants, it was found that the 
group contained one turkey shoot winner and two first " 
runners-up (second place) . It was also noted that none of 
the group with TACACES I training had ^finished in the last 
quartiie; seventh and eighth "place,. Of the 11 subjects in 
this group, there were \eight subjects (72.7 percent)' that, 
finished in the top four-ranks of the turkey shoo t^gon test. 
The iae,a^ f-4 aircraft flying hours experience for this group 
was 333. 6" he jr^. - 

For the TAG ACES II program, seven' respondents in the 
^ sample 1'nc.ic ited that rhey nad completed the traininq. Of the 
"seven sv»b-ec ^s, it\was founc that thirae t|irkey shoot winners 
and two f irs - runners-up (second place) w'eie in this relative 
ly small group." One suojec-: finished in -tihe last qurartile. 
-It was also found the^t six subjects (85.7 percent) of this 
group finished, in the upper three ranks of the turkey shoot 
"competition. The mean F-4 -ircraftr flying hours experience 
for this group was 336.6 hourp. ' Further analysis indicates - 

- 10 4 ■ 

,113 



that the mean Friday GSl> score increased by 28 •! percent 
for the' group with prior' TAG ACE^ I experience. The mean . 
Friday GS I score increased by 36 . 4 percent, foir the group 
with TAG AGES 11 experie^lce. The rtt^n ^Friday GSI score in- 
crea^sed by 38.7 percent for. the total, sample. ' f 

Days S'irxce Last PACT ^ . " ' 

- "DatQ of last Aggressor DAGT Flight: Less than \0 

i30 E^ys , Less than 180 Days ilore Than 

• '180 days /Never . " (D-17) 



All 8Sf' subjects in this study we;re Required to identify 
their most recent Dissimilar Aircraft Training (DAGT) ex- 
per;Lence into three categories:,, less than .30 days, less than 
160 days, and more ^ than 190 days. An additional, ^category, 
"Never," .Was provided for t^hose subjects haying Vo DAGT ^ ' 
experience. Of the 89 subjects, their DAC* experience is 
distributt^d as^ follows: . . ' - ^ 

less than 30 days . ' J^- N = IQ \ -V- 

less than 180 days • ; . ' J^- N =^ 28 

more than 180 days • n =- 14 ' 

Never N = 37 

The relationship of recent DAGT experience and ac 
turkey shoot performance is contrasted in Table 38. It can 
be seen that;^^40^^^^g^rcent of those subjects with the most 
DAGT experience ( < 30 days) were also winners of thq tur- 
key shoot competition. In addition, the«?e same subjects ^ 
(N =.4) comprised one- third of the total group of 12 turkeys ^ 
shoot Winners in the study. The table also shows that more 
than half of 12 winners .had sbme DAGT expedience. * ' ' 

Six of the 10 subjects in the jfirst category ( < 30 
days) were either turkey shoot winners or runpers-up. This, 
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TABLE 3 8 - SUBJECTS PER CATEGORY 





DAYS SINCE LAST DACT FLIGHT 


< 30 DAYS 


<.180 DAYIJ 


>180 DAYS. 


NEVER 


WINNERS 
RUNNERS-UP ^ / 
. THIRD ELIMINATORS 
SECOND ELIMINATORS 
FIRST ELIMINATOIE^ 


4 

■ 3 
1 
0 


2 
4 
10 
8 

4 ■ 


1 ' > 
2 

* - "3 
3 

. ■ ■ 5 


■ . 5 

i 

8 
-10 

10 - 


^ TOTAL 


lo 


28 


. 14 


37 - 



TOTAL SAMPLE 



N = 89 



7 ■ 
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can be contrasted .with the winnerj* and firsT* Ciners-up' in 
the no ex{)/?rience - (l/ever) category. In L-ai^gr- _p, only, 
nine subjects ( 24' ^percent of- the 37 subj^MBfrere turkey 
shfdot YinVienjs ^orv runners*- up, " . ^ ^ 

* ■ - « ■» ' 

Other Visudl Air-To-A ir Simulators Flpwn ' . !^ 

""T" — ' X 

. \ : J ^ : ^ • . • 

"What other visuaj. A/A siiwula tors have you £16wn?" 
(D-18) 

The question was included to determine the extent of 
the subject s experience with other visual aiiM:o-air simu- ; 
lators. As anticipated, the spver^ subjects -hat responded 
io the que"5Kt:Lon concerning TAG ACES II experience 
also responded here, and tl^ey were dele-ed -f rom this analy- 
sis, A tota- 11 respondents. indicated that they had 
f Icwn one f airtiliarization flight of up to 60 mirtutes. dura- 
tion in the TAC simulator for air-to-air combat (SAAC) . Of 
this group, eight of tihe subjects (72.7 percent) had a mear 
F-4 aircraft flight hdurs experience jETf^^G . 3 hours and three 
subjects had a mean of 468.3 hours. When this group was 
contrasze^ with the total sample of turkey shoot partici- 
pants, zhB results were inconclusive. Only one of the group 
was a -urjcey snoot winner^ None were first funners-up. It 
was also foun. that seven subjects (63.6 percent) of the 
group performed in the lowest two quartiles of the sample. 



Combat Experience 



(ft 



\ 

•"How many comoat sorties have you flown? (D-19 ) sorties." 
"What z-s your total dombat' flying time? (D-20) hours." 
"Number of ki:-ls? -(D-21) . " 
"Number of hirs recorded. (D-22) ." 



"Numb^er of 'Si^. en counters. (D-23 " -'^-^ v,<^ 
"N'ur:her ^hostile aircraft enga- ments. ID^^^.*" 
"Number oT hits received/ (D-24 , 

The ^quest^ohs on combat exp£:rience were developed to 
determine^' the degree of relationship between these factors* 

'-*an9-^'iurkey^^shoot per f ormance'^ ( Ei.gh- of the 12 TAC ACES^ I 
.classes responded to the §uestipr,s/ ^ » r. ; 

•There/y^re 18 respondents t:_ tr_rs se-ries of questions. 
A total ';of l7:', respondents had indicated^ fighter cr attack- 

_typ*e as their . aircraft. One respondent indicated a recon- 
naissance-type* 'RC-l35)*and was 'not^ iTicluded here. As a 
group/ the 17 SLjj:ects had a mean, cqmbat flying time of,f 
316.1. hours and a mean' of 13t.2 combat sorties. The'grbup, 
had flown '12 differer : aircraft types rn coiribat. : ais ir- 
eluded six fighter type,^ three attack "vpe. lind th^ee ob- 
servation type aircraft. Result- rndi ate tfiat th-re was - 
one turkey shoot dinner in^tl^^is group ' :: 17 subjects. The 
sub-ect indicated 12^ ::cW:>at flying hours e:^er.ierce in^ 
observation (0r2. OV- 0) aircraft. It /as found tha: thrfee 
subjects finished as ::irst runne^'^-up , ■ ana four subjects of 
the group f inis: ed i: last place". The group was a^sc con- 
trasted with th- predicted rankin<^s of the CIPs wirh simi- , 
lar results. Tie instructors ranked eight subjects :_n the' 
upper half of e turkey s]:i!^ot and nine subjects ir .he" 
lower half (four ranks).. Ti\e results incicate thar for 
this sample, coinbat experience o"f this type is not najor 
factor in predicting turkey shoot performance. 



Own Training Evaluation 



f 



"What is the value of the overall training provided 
in this course to yourself? (D-2 7) . " 
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?his ^question was contamecj in the TAJ ACESi Program 
Evaluation and' Crit;:-qu6 (iee Append/ix . - , * 

"The questionna-.re was aeveloped^esseniially as an end- 
^of-course criti^e fpr^th^ -TAG 'aCES prpgrair^ It consists 
•.'primarily^ of bipolar > descriptiye and acceptabitity scales,, 
^f^rr^tive spacB is provided -for observations anrd other com- 
ments . It" was included in ^ the study to obtain the. sub ject* s 
perceived value of the training th.ey obtained. . ^These data, 
were to be" used to assess). th^ relationshi'p betwe^en the suJd- 
ject's own train:^^ evaluation and turkey shoot performance • 
^hB results from the tota^ sample of 89 subjects show that 
87 subjects (97.8 p^ rc:ent. evaluated the overall training 
as* Having a jpositive effect, and only two of the subjects 



.evaluatedMfhe train:-ng as having no effect on their per- 
:^ormance. In ad^it:LO^, 76 of the 87 sub jects ifevaluated the* 
gaining as having i substarft^al positive e feet on t:-;:=ir 
performance. Both sui: lects who responded c at the training 
had no effect on their performance finished in the slower 
half of the turkey shcot rankings/ and*or.e finished in. last 
/ place. The results of the correlation ^analy sis^, / as shown in 
t'able 27, indicate -he correlation -c^ thi-i: variable with^ 
". .turkey shoot rank, fr:iction.^ wins, instructor pi]ot rank, 
and GSI^scores for* Mcv-vay a: r riday. It za. be seen that 
the *"R" values are qu- :.e . lo; _.:dicating ^ ^^ck of ireiaticn- • 
ship between this Vari^able t:he five c-^endent variables 

. cited. 
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V. PS^CHOME'^.Riq-AND EDUMETRIC OAT ANALYSIS 



.DISCUSSION ' . . ^ . ^ 



; ^ lndivi<^al./aiid 'group perfDrmapce daca were* recorded for ^' 
all th^ 89- subjects in this studyj THe mean gSI performance 
scores for the Monday and the Frid^ data sessions were cal- ' 
culated and plottec^ for each of the 12 classes' and. are 
shown in Figure 19. For these data, two least squares linear 
trend lines were computed, using the number of classes and 
the class mean Monday GSl scores and the class mea^ Friday 
GSI scores,. These trend lines were constructed using the 
datai in Table 39.' • J '^..^ . • 

Four of the 12 TAG ACES- classes in this study were 
subjected to separate analysis. In addition to the normal ' 
TAG ACES Monday and Friday data collection sessions/ GSI 
performance data, were recorded on Wednesday- of the training 
week. This yielded tiiree sets of performance data for each . 
of the four classes. Scatter diagrams, linear and quadratic 
curves, and frequency distributions were constructed. 

For clarification, edumettics is defined he^e as the 
measurement of an individual's gains from training experi- 
ences by the quantitative assessment and analysis of:per- 
foprfance data,, to include individual and group data. Edu- 
metrics is shown to be concerned with measures of learning 
performance in contrast to. pi^chometrics, which is concjerned 
with the measurement of individual, differences (i.e.^ iflpa- 
sures of individual innate abilities and traits) . 



Psychometric ^alysis 

^ the results of the individual performance scores for 
each of the subject pilots in the four-class sample are 
shown by class group in Figure 20. A total of 81 data 
points ^were used to fit linear and quadratic least-^square 
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TAB£K- 39' CLASS AVERAGE GSI DAtA ^ ' 



CLASS ^ 
'NO." " . 



r 

2. 

3 ' 
■4 
5 

7 
. .8 

9 
10 
11 
12 < 



INTERCEPT 



SLOPE 



X = 1 
X = 12 



-3- 



STD.DEV. 



MONDAY - 



\ 



660.29- 
^ 465.^" 
327.1-3 
52^.38 
433.14 
56 7.75 
265.50 
505.88 
341.63 
480.13 
420.75 
377:43 



526.574,6^12 



-12.111,031,47 ^ 



"514.464 



381.242 



-0.3929 



1L1.1445 



FRIDAY 



70;Lfl9 
' '68&.-0V ■ 

66S.13-. 

660.88/^^ 
■ 604.86 

6-52.13 
^583.00 
"576.00. 

558.38 

•671,0|pJ_ 

554.63 

630.29 



688.474,0909 



-9.155,437,06 



679.318 
578.609 



t 



.5 



-0.6382 



.51.7235 



112 



ERIC 



lines for all four classes in the sample. These are shown 
in Figure 21 ;(For clarity of presentation, the individual 
subject data has been grouped by class). When, compared with 
Figure 20, it can be seen that both the linear and the 
quadratic eqxiations developed approximate the centroid of^ 
the mass of data points ^or each pilot. 

Class 7826, as shown by the' data in Figure 2Q^ consis- 
ted of four students, which is half the size dj||the normal 
TAG ACBS class. These individual pilots recel^d more in- 
tense instruction and training due to the lower stu5ent/in- 
s true tojL ratio and the' greater amount * of simulator lise time 
available. The individual performance improvement as the 
length of training increases is clearly apparent in Figure 
20. ' V 

Both the' linear and quadratic lines fit the data weA. 
Objective, measures of these fits are shown in the edumetric 
analysis. The quiacjratic curve is^pref erred in describing 
the d^ta because it approximates true learning rates , whicl 
tend to be non-linear as a function o:^time. Here it speci- ' 
fically shows a higher rate of . learning cfctring the early' 
phases of. training and a lower, slower rate during* the final 
training phases. 

The distributi^ Of the GSI scores by day of training 
are shown characterized by normal distributipns in Figure 22. 
It can be seen that the mean ( X ) GSI scores improved with 
length of training. 

Table 40 indicates that the standard deviation of the 
scores 'decreased as length of training increased. This 
would indicate \f effects of learning. The reduced varia- 
bility in the Weunesday and the Friday Standard Deviation 
values suggests that the subjects w^re using their experi- 
ences gained during the first 2-1/2 days , of training. and 
calibrating their performance responses to the expected 
ancj anticipated performance of the canned targets. 

lis '24 ■ .. ^ 



TABLE 40 - EDUMETRIC DATA BASE 



CLASS 


PILOT 


Y; = GSI SCORE 


NO. 


NO 


MON(X=Q) 


WEp(X=2.5) 


FRI(X=4.^) 




1 


359 


583 ■ 


595 ' 


7826 

•r 


2 
3 


312 
266 


628 
471 


601 . ■ 
589 




4 


125 


508 


547 




1 


309 


494 


493 




. 2 


393 


743 


549 . 




3 


304 


590 


552 


7828 


4 

5 


210 
.531 


635 
> 638 


.794 
447 


t 


6 


234 


332 


562 




7 


304 . 


649 


570 


* 


' 8 


199 


414 


494 




» 

1 


393 , 


546 


487 




2 


687 


617 


851 




3 


391 


522 


739 


7529 


4 

' * 5 


553 
247 


524 

317 


751 

531, 




6 


3^§. 


441 


527 




7 


"577 


469 


"716 \ 




o 

O 


364 


521 » 


581 




1 

•< 


550 


631 


681 




,2 


264 


595 


571 




3 


553 


449 


566 


7831 


5 


- 187 


676 


515 




* 

6 


145 


631 


6:^6 




7 


414 


/59O 


690 




• 8 ■ 


529 


568 


77> 


MEAN ' 


361.778 


547.481 


607.185 


STD.DEV. 


147.563 


101.993 


105.093 " 
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'Figure 20 is included to' show ^he degree of individual' 
change in ■ performance score for eacli subject in this sample 
oVer the 4.5 day training week. The data indicate^ the 
individual subjeqts had a mean performance score (GSI) " 
improvement of 61.3 percent for the 27 subjects in the sample. 



Edumetric Analysis 



The C^SI Wednesday ' performance data collected for four 
of the 12 claares in addition to the normally scheduled 
recordings on Monday and Fr jldSLy are provided -in Table 40. 
The method of analysis was to fit a straight line knd a ° 
quadratic curve through the data. The objective was to 
ascertain the general trend in GSi scores as a measure of 
group learningPrates as the classes progressed. The X- vari- 
able chosen was days o^ training completed. Each student 
was asfeufned to have no training, i.e., X=0, '<fn Monday when ♦ 



r 

when t 

the first GSI scores are" measured. The students were' 



assumed to have received 2.5 days of. trainiri^ (X = 2.5) by ' 
• Wednesday and t»y Friday morning, 4.5 days of' training (X 
\ 4.5).- The Y-variable used was GSI score. 

. . Figure 21 shows a ssy^tter diagram of the GSI scores t 
" A ^^^^"2 of training using the data provided in Table 40. 

The figure -a^o ^ows the linear and quadratic least^^ares 
; curves fit through the data. Both curves can be sefci to fit ' 
; ^ well throu^^the central regions of the data for each day. ^ 

Also,, each->4hows the genera^ trend of GSI core increasing 
with days of training. The -scatter diagraiti also shows the • 
wide varia«i^ in scores for each day and the general over- 
lap which occurs from day 'to day,> This broad variation and 
day to day overlap also points out the general weakness of 
,the predictive ability of the initial GSI Score 

The linear versus the quadradic, curves are contrasted 
in Table 41. Here the actual linear and quadratic equations 
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TABLE 41 - ANALYSIS. OF VARIANCE OF 
LEARNING EFFE ;r.r 



LINEA^R MODEL:. GSI =.376. 345 + 55. 344 ,2 (DAY) 



SOURCE OF 
VARIATION 


SUM-OF-SQ. 


DF 


■ . i> 
MEAN SQ., 




F-RATIO 


SS DUE TO 
REGRESSION 


840,790.0326 


1 


840,790.0326 


56.-894,993,72 


SS ABOUT 

REGRESSION 

(RESIDUAL) 


1nU67,456. 189 


79 


14,777,926,45 




•TOTA^i SS 
ABOUT. MEAN 


2,008,246.222 


80 







R (Coefficient of Determination) = 0.418,668,798,3 

% '(Multi^^ Correlation Coefficient) =. 0 • 647 , 046 ,210 , 4 



QUADRATIC MODEL: GSI = 361. 7 + 9.8 . 964 (DAY) -9 


.873,3 (DAY) ^ 


SOURCE OF 
VARIATION 


SUM-pl>F-SQ.* 


DF 


. MEAN SQi 


DAY: 

MON. = 0 
rWED. = 2.5 
FRI. = 4.5 


SS DUE to" 
REGRESSION 

SS ABOUT 

REGRESSION 

(RESIDUAL) 


88-4^476.7408 

. • v 

1,123, 769 . TO^ 




442,238.3704 
14,407'.30l",04 


TOTAL SS 
ABOUT MEAN 


2,008,246.222 


80 





R (Multiple Cr ■ L-ion Coefficient) = 0.663,643,322,6 
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are shown along with an analysis of .variance table for the 
linekr regression and "variation blre^down" for the 
quadfratic equation'.:. The multiple correYation coefficients 
(Rf ^re also pro\?^Sed as well as coefficients of determina- 
fion (R ) for b9th equations. The F-ratio for the linear 
I mo'del is included and is -significant at the 99 . 9 percent 
level/ (F^gg'g (1,79) = 2rl.68) . This indicates that the slope 
of the straight line is significantly greater than zero and, 
thus, that GSI Score increases at an average rate of about 
55 points per day of training over the 4-1/2 days of train- 
Nlng. . ; 

^ J 2 

The calculatiort of R- (the coefficient of determination 
or ^e multiple correlation coefficient squared) . is a mea- 
sure of the proportion of total variation about the mean of 
the GSI Score explained by the regression line. Thus the 

Straight line explains about 42 percent (R = .419) of the 

— > 
variation and the quadratic equation explains about 44^ per- 
cent (R^ = .440) of the variation between training time and 
improvement in GSI.\^ ^ 

A test was also 'made for "lack^of fit" of the straight 

line to tSe GSI Scores. The teat involves Jbreaking the 

residual siam of squares into two\ parts, one/part measuring* 

pure error '*and the other measuring lack-of-*f it . Repeating - 

the residual gum of squares for the straight line in Table 

7 

41 results m the following breakdown: 
SOURCE OF .' ' 

VARIATION D.F. SUM OF ^SQUARES MEAN S.QUARE '^^'-v^ RATIO 
Residual « 79 1,16 7,456.189 ^ 
Lack-of-Fit , 1 43,686.708 43,686,708 3.032,262 

Pure Error 78 1,123,769.481 14,407.301,04 i; 

9-^3 (1,78) = 3.92 ^ ^ J ■ 
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Draper & Smith. Applied regression analysis . Ndw York; 
John Wiley and Sons, 1966, 26-31. 
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Now sir.ce 3.032''^< F 95 (1,78) =3.92 there is no .reason to - 
> doubt the adequacy of .the linear model, i.e., the lack of 
. fit ds not^signif icant. 

1 A fyrther point of interest is the actual .normality of 
the distributions of the GSI scores being an^yze^ by day, 
that .is, is there' any reason to do.ubt that a given set of 
scores i^ normally distributed? The Kolmogorov-Smirnov 
(K-S) ' te^t of goodness of fit was applied to GSl scores 
for each day. ^ The scores were found. to be normally dis- 
tributed at the percent significance level for each of the 
three sets of GSI ^cores. 

• Since it"^ ha^ been ejStabliaJaed that there is no reason at 

the 99 ppro^n^Xievel to doubt that the GSI scores are nor- 
mally distributed, it is reasonable to- present Figure 23 
which shows three normal densities with parameters (means and 
standard deviations) equal to their estimates calculated from 
the' GSI scores for each day. This- figure graphically shows 
the changes in T5SI Scoreydisttibutions which take .place 
during the course of yraining. The means of the distribu- 
tions increase with/training time. On Monday the standard 
ieviation of GSI scores is compared to Wednesday and Friday 

^creased about .31 percent oyer -Monday (IS (Wednesday) =102 , 0)^ arid 
then by. Friday there appears to^be a slight increase, (S(Fri- 
day) =105.1) . To determine ^statistically if these- dif-i \ 
ferences in variance exist, Bartletts chi-square . test^ fpr 
^quality of standard deviations from normal distributions ^ • 
4as applied. ; It was determined th^ the null hypothesis of 



/(S(Monday) = 14/7.6). . By Wednesday , however this has de- 

31 perc€ 



2 2 

no difference between variances, (H : a (MONO = a (WED) 
2 o 



0 (FRI4 , canno^be rejected at the 95 percent conf id^ce 
level but can be rejected at the 90 percent confidence leve^. 



8 c . ' ' - 

Ostle & Mensing, Statistics in research (5rd ed.). 
Ames: - Iowa State University Press, 1975., 489-490. 
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Ostle^and Mensing. Statistics in research (3rd ed.) . 
Ames ; Iowa State Univ;^rsity Press, 1975^ 127. 
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VI. CC»ICLUSIONS AND RECOMMENDATIONS 

, L\ ■ ■ ■ 

General v 

•An experimental investigation has been performed that 
statistically validates the ability of anT^meirically de- 
rived performance, measure, the GSJ, to correctly predict 
student pilot performances in TAG ACES I free engagement 
exerdises. The empirically derived GS I is shown to exhibit 
co^rrect prediction capabilities of student pilot .performance 
comparable to that of 'expert opinion, subjective stud^t 
performance predictions l^y instructor pilots, - 

„ The empirically deri\|ed GSI predictor was improved' 
usipg statistical methods^ The four parameters^of the ini- 
tial (einpii;ical) GSI, when optionally weighted/ were shown 
to predict student tritlot placement in the turkey shoot with 
about 75 percent accuracy. Thesle- four parameters, time in 
gun firing envelope, average mil error, offensive/defensive 
time^' and time to first kill, are intuitive to the - experi- 
enced combat pil^j^ as measures of ACM skill, Eafch of ^jtjie- 
'^four^: when objectively measured, can bemused as teaching aids 
in the development of air; combat skill in the student .pilot, 
J'urther , improvement in the GSI was ^obtained »by includ- 
ing 'certain available objective and subjective parameters, 
Th^ optimal methods are shown to be excellent predictors^^of 
student performance (at least within the experiment data) 
showing probability of correct student performance predic- 
tion near 80 percent in free engagement exercises. 

It is specifically recommended that the GSI algorithms 
and methodologies of this, initial study be tested in the 
Simulator for Air^to-Air combat (SAAC) at Luke AFB and on the 
Air combat Mcmeuvering Instrumentation -(^MI) Rcmge at Nellis 
AFB to determine an objective measure of transfer of ACM 
training between the simulator and tlie aircraft, 
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IncjreaSed Sample Size 

The results of the study yield* GSI models that may be 
applied to tjie TAG ACaS I population. . The 3'i:\ple size use.d 
to derive these mode/s\was ^relatively snail ,(12 classes) but 



was related to tlief wholiBby statistical inference-. It is 

' ^ I ^ - 

desirable to continue daza collection and, statistical analy- 

ses under the^ame Goa^trol conditions as the* experiment to 

accumulate a larger data sample. - 

It would be useful to collect additional TAG ACES I 

data for the following reasons: - . . • • . 

1. To provide a*larger sample which would provide 
more precise information on the distributions o:^ the data 
being considered ; r ' 

2. To validate the' predictor models derived in this^ 
stfudy. Careful examination of GSI data Collected previous^ 
to this stu^y was found to be poorly dociamen ted* and of 

^limited u^je^in validating, the predictor models. ^Care must 
be taken to assure that reasonable controls are placed on 
the data collection itself as-iack of control? affect the 
validity^ of- the samples themselves:. By i.ts very nature, 
this kind of dat^ is very sensitive-i-^ .Lack of careful -^ampl- 

^ ing can result in collection of data from essentially 
different populations than tha.t desired and, hence, valida- 
tion becomes dp.f ficult. . . ' \ - 

Demographic Data Correlations . 

The master data base provides a means for further 

Statistical .anal > v/hich can be oi valu^ in assessing 

^ ' ^ ' ■ ^ 11 

training and tr^iinjing requirements in ACM simulators. 



^^On file at Vvght Corporation, Dallas, Texas. 
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It IS recommended that an investigation^ be initiated ' to as 
certain what demographic correlations can contribute, to th 
overall readiness training program. In order to accompiis 
this objective, it "is necessary to continue to at) collect 
these data, b) supplement these data with other iata which 
may be of value, and c) analyze the data to obtain correla 
tion with simulator performance, measures and, ultimately, 
,d) assess performance on the ACMI range exercises. 



1 - 



Apply GSI to Othi^r^CM Simulator .TraiViing 

The parameters comprising the GSI, if meas ured ' in a 
similar manner and "jander similar conditions, are applica- 
ble to other ACM simulator training. The interrel^'tion- 
ship of these parameters, i.e. , weighting and interaction, 
is believed -to be spe^jific for a. {Particular simulator and 
training syllabus, ^ Jt is recommended that the GSI, as 
derived for ,TAC ACES I, be introduced as a prospective 
jneasure of student pilot performance 'in an ACM simulator 
such as jthe SAAC and adjustments made in the parametric 
contributors to develop a statisticLally derived GSI specifi 
to that facility and training syllabus. 

The GSI Application to ACMI Range ■ ^ > - 

^ The promise' of the GSI as a screening tool to aid in 
♦the selection of^ fighter talent is premature, but given 
a larger data sample and successful application of the GSI 
I^Q range operational exercises such as the ACMI range at 
Nellis, the GSI could become that powerful tool. 
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Potential Utility^ of the GSI 

The GSI was shown to '-be a measure of student pilot per- 
formance in the TAG ACES I Program. GSI scores -indicate 
the re lati^ve^ performance of students in, the simulator and • 
careful scrutiny ot the GS'i contributory parameters can 
evaluate the stror/g ^nd weak points of a given student rela- 
tive to his overall performance measure. These "part scores" 
/are associated with ba'sic flying maneuvers, tracking, wea- 
> pon, switchology , etc. fr^m* which judgements may be made by 
the( instructor pilot where to concentrate his training 
efforts. , ' , ^ 

The GSi may also be utilized to obtain a measure of 
student pilot learning trends during the simulator train- 
ing period. The' skills of.'pilots <in air. combat can 'vary 
^ greatly depending upon individual background experience 
and innate ability. The individual learning ^bili'ties 
also vary. The GST may b^ used as an indicator of. a 
'*'pirlot*s current prof iciency in air'combat , as^well*as an 
indicatoir of improvements in air combat skills in the 
simulato'r. 




The GSI can *be j{ised to establish' optimal training 
period .for the'^norm student by statistical investigation of 
initial stud^ent skill -and skill grgwth over training 
periods varying in duration. "A cursory survey the 
12 class sample in this experiment indicates that an 
opJtimal training period in, the simulator can i3e established 
for the TAG ACES I , population by .further statistical analy- 
ses of ^tudent entry skills and student learning trends. 

Contributing parameters that comprise the Air Combat 
Simulator GSI have rudimentary commonalities with many other 
flight simulator training devices. It is pr'obable that 
other flight simulators,* i.e., Weapons Systeip^ Trainers 

(WST) > Operational Flight /Trainers (OFT) ,"ri:nst rumen t - > 

* / 
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Flight Trainers (IFT)yetc. , can utilize the same.or simi- 
/l^r^metho-dologies "Resented ih' the report to achieve ' 
, -ccSmpairable simulator performance measures. . 



Uciiity of bata Taken During Turkey Shoot 



/ 



. • • . " The i;urkey shoot Idata were examined .to investigate .the 

. - .,.-.. u|ility of the -dataSollected . during turkey shoot coitipeti- 
f 'tion. The per formance'' measures and the data formats- were 

■ :\ -^essentially .'identical to those- used in €he QSI •data. A . 

J -v. '^basic^ difference is that per-f onfiance^data were recorded ^ ' 
separately, and simultaneously f or ' each^ pair cif combatants . 
^.No .GSI scbries '.were^computed from, .this data ^et. ^ . 
r"i > . . ■ ■• v v^.^^, performarice results Were' e:<a|ru.ned for a class 
' . ■ i^;. ^selected at random^; ^.."^ - f 'T . 

• ' ,• ^ The d^td indicated th^t pilots who- finished in. the 

■ , , upper, half .'of ^^^^^t^^^ as . a group, Ipwer mean 
..• ' ; . :t mlhimum altitude values " than $i lots who finished in the 

• v , lower h^lf .. -of' the turkey shoot. The data sh^w that a suffi- 
■;>■>; c.ient body of -pilot -^^ has been collected. to 

■ ^ , . ^^^^^t a detkiled^statistical Analysis. A cursory examina- 

• of the data indicates that trends of a relationship " 
^appears >4:o" exist between turkey shoot rank and factors such 

.. - -as maximum g, minimiam altitude/ and ' of f ensive time. The 

* ' . engagement data may be of. value since they approximate 

engagements.- on an air combat maneuvering' range.^' The data 
^ jnay also .be useful in determining links between GSI per- ' 

formance predictors and those predictors to be .determined .• 

• ^' . for the A(bll range (s). 
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Limitations of the GSI System 

The g5i as presented in this report is .specific to 

■ . r ' ' ' ■ * - 

TAG ACES I traiihiing. Howftver, its application, to other 

air comb'at simulator training where, the envi.ronmer^ital train- 
ing features are similar, i.e., training hardware, software, 
.and traioing ,syllabuises are of a similar character, may be 
expected to yield good measures of air comlpat skill (in the 
simulator) ."^ ' ^ ' 

^ The GSI scoring system is derived for air combat one^ 
vers-^us-one engagements at the inception of • of \!fensive/def en- 
sive maneuvers. In its present form, the GSI is not appli- 
cable where initial sightiiT^ of adversary or two-versus-one , 

^^ii^ one-versus-twp, is -instrumental in the training scenario. 
The GSI is an objective indicator of air combat skill 
in the simulator but should not construed as .'an absolute 
measure. It is .not proposed as a substitute f or . sub jective 

- opinion.. When the. two measures, Gsiv-^J^d the subjective 



opinion of the instructor pilot are used" ±ti conjunction, 
they produce a. maxima'l evaluator of air combat simulator 
skill. ' * . . 

^ ■ A 

■ ' ' ' 

GSI Application to Other ACM Facilities 

The de'cjree'' of- fidelity of simulation , training sy 11a- - 
bus and the extant of training are factors governing trans- 
fer of training for a given task. In general, ACM simu- 
lator facili^ties differ widely in the synergistic fidelity • 
of air comi^cit; ^ . 

Lac;^ of absolute fidelity in a simulator requires-, the 
student pi lot to suppress many preconditioned responses 
and acquire associated responses'" to representative exter- 
•nal stimuli. The ability of the student ijo transcend to 
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this representative environment directly affects his per- 
fojgnance in a .particular simulator. 
• ^ The differences in fidelity of simulation between simu- 
lators of like. kind and the difficulty of association trans- 
fer experiphced by the student will determine the applica^ 
bility of the GSI to other ACM simulators as a measure of 
.ACM skill and as « predictor of free engagement one-versus- 
one contest results/ 

SoKm ax^amples of known ACM simulator fidelity differ- 
ences which can influence GSI application are motion/no- 
«otion, g-suit/g-«eat, ground rush visual cue, and the ex- 
tent of ooii|>ut«r wodeling of aircraft flight characteristics 
(aerodynamic fidelity, control response fidelity, instrument 
fnd weapo« system fidelity). The effect of the differences 
can b«' positive, na^ative, or neutral, on the Contributory 
paranJiatersl of the GSI, 
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APPENDIX A - ANALYTICAL METHODOLOGY 



The analytical methodology used , in the study began- With 
preparation of elementary statistical displays of the GSI 
and the four component variables used to calculate the GSI 
score. These displays consisted of histograms and scatter 
diagrams. Variance-covariance matrices and correlation 
matrices were also generated to analyze relationships be- 
tween the variables. 

Regression analysis wa6 used extensively in- an attempt' 
tfo define suitable functional predictive relationships be- 
tween, the various candidate predictor variables and turkey 
shoot outcomes. 

Two Y-variables (dependent variables) were cqnsidered 
in the regression analysis. They were turKtey shoot rank, 
I.e. ,\l,2, 3, 4 and fractional wins. "Fractional wins'!, is 

defined as the ratio of total wins to total engagements in 
the turkey shoot for a given participant. 

Both va^riable selection and ridge regression w4,re used 
in addition to all-variable regressions to explore the 
utility of direct predictive relationships. Various non- 
^^_^near relationships (in the candidate predictor variables) 
w^e explored, but none provided relationships as good as . a 
simple GSI ranking predictor. There are several possible 
reasons why this was so: Exploration of the X'X matrices 
indicated that in all cases minimum eigenvalues were very ' 
..close to zero. This is" indicative^ of the .existence of . } 
multicolinearities in the predictive variable sets.' This 
. condition indicates that basic assumptilSns generally used in 
the application of least squares are being violated and also 
that it is- likely the parameter estimates will vary sub,^ 
stantially from sample t£^ sample. Another difficulty was . 
shown to exist from the analyses of variance performed. 
This waa-the significant .variation de;6tected between, classes . , 
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The regression models were obviously affected by this and 
the fact that no constr'aints Wc:re (or could be) applied to 
rank predictions. For ^ example/ only one winner is allowed 
per class, but seve-ral might be predicted. 

In' general, models ejcplored ^using ridge regression - 
showed a degeneration in predictive capability as the bias 
factor was increased. ...While, in gen.eral, the parameters di-d 
stabilize, as might be expected, the predictive rates de- 
clined and remained una:cceptable . * 

The all-variable; variable selection^ and ridge regres- 
sion programs used-'in. this study had been developed by 
Vought previous to the' beginning' of this study. 

As it became apparent that' the regression programs were 
not providing useful indicators of predictive ability, it 



was decided to explore three sources of variation in the GSI 
scores and the GSL^ component variables. Using basic analy- 
sis of variance mj^hodology, the sources of variation inclu- 
ded in the three-way analysis were "between" days, "between" 
classes, and "between" turkey shoot ranks. "In general, 
significant differences tended to appear between days and 
between classes. A 

At the beginning of the study, a master data base was 
designed and then implemented. This brought data from the 
source- data tapes into a common file where it could be con- 
veniently studied, manipulated, and reduced to foifms , suita- 
ble for use with th^e statistical programs. 

The. next and final sratistical program exercised against 
the data was the Disq-irrfinant Analysis - program provided in 
the SPSS package available on Vought's System 370. Discri- 
minant analys'rs can be used to classify , data spts (''^into ; pre- 
defined groups. In the case of thisy ar^^lys is , the - groups ^ 
were defined as combinations of turkey- shoot ranks. As 

•4 ■ 
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explained in the main body of the tex-t, -this part of^the 
analysis was" performed for four different .group defini- 
tions with four_different data sets. ' The program was 
always^perated in the variable selection mdde using the 
LamlJ^da variable selection^ option. Data sets'," prior to in- • 
put, were so^rted by turkey shoot rank ^ith all winners at 
the .top of the list, runners-up following, and so on.. Pro- 
gram control parameters were then used to define the number 
of groups and th^ number of members of' each group. As noted 
above, four groJupings were defined for four different data 
sets. . Thus, in all, 16 discriminant airalyses were per- 
formed. These,. in general, provided the best predictors of" 
turkey -shoot outcomes developed in the study. The results 
are documented: in^ the main body of the report. ' ' 

Several other cqmijionly used -statistical techniques 
were also employed. Among these were the calc'^lation of 
confidence intervals "on the proportions of correct classi- 
fications of cases by the discriminant program using data 
from the 12-class sample. This procedure made use of the 
normal approximation to the binomial distribution^ which 
is often used where sample size is' adequate. ' Certain tests 
of hypotheses were also used during the comparison of the 
^disdriirtin^ant results calculated from the 12-class sample 
and with^the four classes of data -used to test various pre- 
. dictors. This was used . to test equ^ity of .prediction rates 
of the discriminant predictors on the i2-class ^.experiment 
data with the four class test data. / 

Certain other tests were employed to test for normality ■ 
of data and applicability of a straight line to the learn- 
ing rate data used in the edumetric analysis. Footnotes 
,are used in the main text to identify references applicable 
'to the statistical methods employed. ' 

... . .■ . ^ ^ . 
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• . • ' TAC ACfeS BACKGROUND SURVEY ^ 

1. FULL NAME ' RANK -^D-l) '2. dItE ' 

3. CLASS & PILOT # 4. ACES I ACES II □ 

5. MIL ADD. SQDN (D-2) WING (D-3) .BASE (p-Q) ZIP, (D-0) 

6. TOTAL- FLYING TIME (D-4) 7 . TO^ FIGHTER TIME (D-5) 

- 8.. TOT. F-4 TIME (A/C & IP) (P-6) 9. SORTIES (LAST 6 MOS) (D- 

10. CURRENT IN: F-4C □ , F-4Dn . F-4E □ , (OTHER) j D-8) .fl 
■ 11. PRIMARY DOC: A/a{J , A/gQ , RTU IPQ , (OTHER) (D-9) R 

12. RECENT BFM/ACM EXPERIENCE : SORTIES-TOTAL (D-10), 
LAST 6 MOS (D-11 ) , LAST MO. (D-12) 

13. TIME SINCE LAST BFM/ACM: 0-2 WKSQ , 3-4 WKSQ] 
5-12 WKSD ,13-25 WKSD , 26-52 WKSG (D-13) 

14. WHAT A/A MISSILES HAVE YOU FIRgD? AIM-TQ , AIM-gJj , / 
AIM-4n , NONEn (D-14) 

15. ARE YOU AN FWIC • GRADUATE? YESD , NOD (D-15) t"^'^ 

16. HAVE YOU PREVIOUSLY ATTENDED: TAC ACES iR 

TAG ACES IlO , NOD ' (D-16) ' - 

17. 'DATE OF .LAST AGGRESSOR DACT FLIGHT.; * LESS THAN 30 DAYSQ 

LESS THAN 180 DAYSTI^, MORE THAN 180 DAYSQ 
NEVERQ ^ (D-17) 

18. WHAT OTHER VISUAL A/A SIMULATORS HAVE YOU FLOWN? (D-18) 

19. COMBAT EXPERIENCE: YES □, NO Q. IF YOU HAVE HAD COMBAT 
EXPERIENCE, WHAT IS YOu"^ TOTAL COMBAT FLYING -TIMe)? .(D-20) 
HOURS. HOW MANY ^OMBAT SORTIES HAVE YOU FLOWN? (D-l9)' ' 
SORTIES. .WHAT TYPE 'OF AIRCRAjrl" HAVE YOU FLOWN IN COMBAT? 
(D"0) NO. OF APRCRAFT ENGAGEMENTS (D-24) . NO. OF HITS 
RECORDED (D-22) . NUMBER OF HITS RECEIVED (D-25) . NUMBER 
OF KILLS , (D-21) . NUMBER OF SAM ENCOUNTERS. (D-2 3) . 

20. DATE OF BIRTH (D-0) . 

*D-0 - NOT • JI^JCLUDED IN ANALYSIS' Vj 
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TURKEY SHOOT FORMAT 

fILOT NO. . ' DOUBLE ELIMINATION 

' ' STUDENT PILOT BRACKET 



> 



1^ 3 




> 



> 




SEMIFINAL 



SEMIFINAL 
l,OSER 
5 



1ST ELIM 



> 



LOSER \ RUNNEpUP 

2NP CUM X 1 y x ^ ' 



ID ELIM ^ 



• 1ST ELIM ^ ^ * 



> 



2h|D ELIM ^ 





^ * 7 



1 



WINNER 
3 
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^ INSTRIXTTOR OPINION FORM 



In your( opinion, iiow vill each of the students in clasa 



perform In the Turkey Shoot Competition? Please rank-ortier the atudenta. 
on a scale of from 1 to 8. Uae the rarik of 1 to identify the student vho 
you fi^el will vin the jTurkey Shoot, the rank of 2 to identify tike first 
runner-up, i^hd so on until the rank of 8 to identify the student who you 
feel will place last. Please rank all the students. 



INSTOUCTOR PILOT DATE 



■ ^ 

NOTE: Please complete this form before the student Turkey Shoot Competition 

each Friday, atie form will be colli/ctcd from you t^y to^ R.A. Jorge sen 

t ■ 
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• TAC ACES DATA COLLECTION '/ERIFIGATION 
CUSS •' DATE ■ 



K^THE CURRENT. TAC ACS' TRAM SYLUBUS WAS APPLE!), AND ADHERED TO,"CONSISTEmY DURING Tl DATA. 
RECORI)ING.PERIOD(syFOR THIS CLASS. /' • ' ' 



INSmCTOR PILOT / " "m^ 



2. THSffiraffiNO APPAm mNICAL^ORELEmm 

^ z ■ m DATA recordi/g mmim .ms 'cuss., (toions as noted on reversfside of THis mr, 



INSTRUCTOR PIEOT ,. DATE 



■» ■ ' . . 

. ■■ • ; ■ ■ ■ ■■ . . ^' . ,. ' ' : ' , ' . « : ,. 

■ ,, ■ ■" * ' i / ' 

., J.; . THERE m:^?^I(^ft|COMmffl J^^^ OR ANNIES. M 

, ^ FOR THE CUSS,1ICH WOUID kcT THE M BEING CoJlECTED.'' ■ lEXmiONs''AS NOffl) ON REMB SIDE Of' 

' ; -.THis m) • ^ • ' ; ■ ■ > : 



.•:|ilEr;'COMPLETE])'P0^ '.' ^' . : ; ■ \ i ^ 

. ' ;MR; R.;A. JORGENSEN. ^ " REAL TIME COTOG ■ .' . ■ ' m 



TAC ACilo PROOrxAM EV/JJUATION AND CRITIQUE 

CLASS # ' PILOT # 



DATT 



XAC .AC53 PSOGPJ^: I ( ), II ( ) 

HOTSr-'^ls evaluation vlU be conducted in three parts. 

In part I you are asked to give your ratings of the utility of this training 
concept . ) In short, voull regular jexpcsurc to visual air-to-air simulation be 
beneficial? Does it possess the potential to increase your coabat capability? 

In part II you are asked to assess end rate the relative benefit of the 
simulator itSifilj; including instructional features. What inprovements must it have? 
Wiere is it' good enough? 

Part III consists of unstructured questions relating to simulator trtvlning 
capabilities iCnd limitations, course value, instruction, and the TAC AC£S program 
in total. * ^ 

FART I: 

Use the foUowihg scaleci tO; rate each (iuestion end add appropriate corraents 
^"hen necessary: . - ' 

gating ' General f't^aning 

5 Substantial positive training ^r^- 

k Slight positive traiiClng 

• ' 3 No effect^ 

2 PDssible negative training 

1 Definite negative training 

A. What is the value of the overall training provided in this course to: 





5 


4 


3 


2 


1 


- C(M'SNT 


Experienced pilots 












\ ■ — 


.Inexperienced pilots 












Yourself 












A/A DOC pilots 












A/O DOC pilots 












RtU IP* 
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3. How did this trair.ing affect your knowledge or prof-iciency in the follov^-ing teckr:; 
.Use rating scale on page 1. ^ 



s 


5 


1* 


3 


2 


! 1 




Sn^agcnent Gccr.ctry 












r » 


Includes visual slar.t rar.re, 
aspect deterniinatior., closure 
rate control, etc. 


















Includes st^.tus itonitoring, 
leftuich envelope, la^xnch * 
constraints, ' etc. 

• > 

i ^ 






• 


AIJ^-9 liLplo^^ent'. 












Sec above 




> 


Gun 5:nvelcpe and LCOSS 
-^^^ — I — ^ 














< 







\ 
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PART II 



The slaulator' s chief purpose is to aid the instructor in teaching various 
air-to-air tasks. As such it should be like the aircraft in znany respects but not ' 
necessarily in every detail. In addition it should be design d to ease the workload 
on the iastructtor while still providing effective control ove^r^^the^^ei^ 

A. Cccpsre the sitiulator to the aircraft in the following areas using the rating 
scale provided: 



5 - Much better than 
r U . Slightly better 

3 - About the sazie 
' 2 - Sliglitly worse' 


air 


•craft 


* 


• \ 


5 




i 3 


2 


1 




' Accelpration Performance 


!" 




!■ il 

J; li 






Deceleration Perforaance 






1 
1 


1 






Roll Perforcaace 








i , 




1 


titch Perfonaance 




> 


I 


1 - 






Yew Performance 






i| 






Turn Rate 






II 

1 !i- 






Aa\ Indications 
(buffet, tone, noise) 














longitudinal SticK Feel 








1 




* ■ "■ ■ H- 


Lateral Stick Feel 






't 

1 






Rudder Feel \ 














AIM- 7 Pterfomancc 














AIM-9 Pcrfomance 












V 


Gun Performance 














Gun$ight Berforcance 














\ 

IR Tone Operation 




i 












B. Do you feel cockpit notion is necessary for an A/A ciriulator? Yec ( ) No ( ") 
tCocsient: . " . 



PART III ^ : 

A. What A/A tasks and/or Bri r-n^uvers CA^ be trained in the cibulator? 



B. What do ybu consider to be the best training features of this siaulctcr? 



C. What A/A tasks and/or E?!'! maneuvers CA?:?IOT be trained in the sinulator? 
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What do youysonsider to be the most significant limitations of this simulator? 



II4S the training provided during this 
fighter skills? Yes ( ) 

CoECient; 



week improved your overall operational ' 



Should the course he offered on a recurr: ng basis? Yes ( ' ) No ( ) ^ 
Coriment: - f 



Connent on the quality and quantity. of instruction. 



What features/capabilities would you like to see ^ded to this sinulator? 




Lict'eny cornents/recor^nendations you have rcgardinc the TAC ACES progrea. 
(i.e.,* syllaous/ad'^dnistrative/scheduling/quarters/transportation/etc. ) 



1 



T 
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f APPENDll^ C ( 

tag aces i training syllabus 
;and 

turkey shoot competition rules 



TAG ^ACES I 
SYLLABUS 



DAY 1 



Sortie #1 - Simulator Familiarization (:30) (F-4/F-4) 

Objecrive: Jo become familiar wi'th simulator visual 
display, switchology, aural and dynamic cues, flight 
controls, cind performance characteristics. 

Pilot will perform following tasks: 

a. Acceleration maneuvers ^ 

b. Rolling maneuvers ' ^ 

c. Turning maneuvers " , 

d. High and low altitude flight 

e. High and low speed stalls * ' 

Sortie #2 - VJeh- jns Familiarization (:30) (F-4/F-4) 

Objective: To become familiar with AIM-7E, AIM-9J, 
and 20inm employment. 

. 'pilot will^ perf orm/demonstrate following tasks: 

/ ^ 

^. AIM-7 and AIM- 9 employment against a con- 
trolled target 

b. . Gun ti^acking exercises against a contr'olled 
^ targel 

c. Undj^ritanding of weapons switchology 

d. Itedogriltion of aspect angle, range, and 
^^^^ ' cflo^^re velocity. 

e . Maj^' perf ormeince maneuvering 

I 

Sortie #3 - Performance Measurepient Data (:3O0 (F-4/ 

Cqmpvtter Flown Target) .25^ 



Objective: To collect a bafseline' perf ormance mea 
surement.on each piloU as he flys against a pre- 
recorded prpfile. ' ^ . ' ' 

^ The pe^f. ormeince measurement will consi^s;^ of the . 
following exercises: 

a. 2 X Stabilized (Cine) tracking exercises 
• b. 3 X Head-On maneuvering exercises 



-?- 
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DAY 2 • 

— 



Sortie #4 - Gun/Tracking (1:00) <F-4/F-4) 

Objec'tive: To fully understand operation and em- 
ployment of gun and LCOSS. '1 



\ - Ob] 



pilot will accomplish: 

a. Stabilized tracking exercises 

b. High angle gun emplci^yment 

c. Tracking a maneuvering target 

. Sortie #5 - Basic Figfiter Maneuvers - Offensive (1-00) 
j (F-4/F-4) 

Objective: To understand and be. able to perform 
basic fighter maneuvers from a canned set-up. 

Each pilot will perform the following: 

a. High" and . low ^Yo-Yo 

b. Quarter plane maneuver 

c. Lag roll 

^Acceleration and separation maneuvers 

DAY 3 ^ , 

Sortie #6 - Basic Fighter Maneuvers - Defensive (1:00) 
(F-4/F-4) ^ . , 

. Objective*: To understand energy irianagement 'and bas 
defensive maneuvers. 

Each pilot will u»d'erstarid and practice: 

a. OverahcSets 

b. Extensions 

V c. Reversals : 
- d* Jink-outs 

Sortie #7 - i^ir Combat Maneuvering - Similar (1:00) 
(F^,4/F-4) 



Objective: To increase ^proficiency in entire 
maneuvering envelope. - . ^ 
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Each pilot will demonstrate understanding of: 

a. Use XDf the vertical 

b . Lead turn . . 
' c. High AOA maneuvering 

d. Combat separations 

DAY 4 ^ " ' ^ . . 

~~ f . 

Sortie #8 - Threat Orientation (1:00) (F-4/Threat) 

' : . ■ " ■ 

Objective: To develop an appreciation for. the per- 
formance characteristics of atypical -threat air- " 
craft. ' o ^ 

Each pilot will observe the following threat charac- 
teristics: 

a. Flight control responses ^ <r 

b. Turning capability 

^ c. Performance envelope (altitude , airspeed, 
etc. ) ' / ^ . V 

Sortie #9 - Air Coif±>at Maneuvering - Dissimilar (1-00) 

(F-4/Threat). ^ * / . 



Objective: .To 'increase proficiency in maneuvering 
agai/nst dissimilar aircraft. , ' 

Each? pilot will fly eadb aircraft in fluid engage- 
ments against each otheri. Lessons learned will be 
^ discussed during debriefing. 
DAY 5 - ^ 

Sortie #10 - Review of" Sorties, 1-9 (:45) tF-4/F-4) 

Objective: Briefly review all previous sorties for 
areate of confusion/misunderstanding. 

^ ' Each pilot will demonstrate knowledge of basic con- 
cepts of air-to-air combat maneuvering. 

Sortie- #11 - Performance Measurement Data (:15) (F-4/' 
Computer Flown Target) 

Objective: To collect an end of course performance 
me as lire me n't as the pilot flys against a pre-recorded 
profile. ( / 

. ' ■■ • . 
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The perfo^rmance measurement will consist of the 
following f ^ ' 



f a. 2 X Stabilized (Cine) tracking exercises 

[ b. 3 X Head-On maneuvering e-xercises 

I Sortie #12 - Turkey ?Shbot (F-4/F-4) ' ' * 

* ' '\ Objective: To allow. pilots to demons tr ate ^thxeir 

^ir-to-air ability in a class fly-off. 

Each pilot will be eliminated after losing to two 
other pilots in a double elimination tournament. 
Rifles of engagement will be briefed prior to st^rt 
^- of fly-off. / 



On all 1.0 hour- sorties , pilots Vill switch cockpits 
after first 30 minutes. 



-/Sorties should be recorded for debriefing. 




■ TURKEY SHOOT RULES 
Doub le E limin ation 



•^Initial pairings will be made by dr^ijig names from, 
a hat. ^ / 

' ' ' ^ ^. . ' 

Both .aircraft will be F-4E's at 15,000 feet and 425-. 
kts , head on at 18,000 slant range. 

Paired participants will flip a coin for choice of 

cockpit . * ' - ' 

There will be a 3, minute time limit for each engage- 
ment.. . After 3 minutes, both aircratft wij.1 be reset 
to the initial set-up. 

Aircraft over-G (10 G's) , hit^g the ground, and ' 
spins that bomb the computer 5ife automatic kij.ls.\ 

Head on gun kills are not authorized. An aspec|: angle 
greater th^n 135 degrees for the shooter at time of 
Hill is cor^*sidered a head on gun kill. 

Radar . lock-on can only be accomplished ty pilot acti- . 
vated auto-acq after the second "engagejnent . Radar 
missiles will not be used until thg, third engage- 
ment. > • 

Switchology trickology is unauthorized. 

Entry fee will be decided by the class (normally $1/ 
pilot) . 

These may be agreed^, to or changed by the entire class. 

liie, cheat, and steal, but keep your six clear and may 
the better man wini! ^ l- ^ ^ ' 

Head-on kills on the initial pass are not authorized 
at any time . 

\ • - • 



LIST OF ABBPEVI:?^']; IONS ^ , ACRONYMS AND SYMBOLS 



\ 



• A/A , ' ' • 
. A/C 
' ACES 
ACM 
ACMI 

. 'afhrl ^ 

AVG 
BFM . 
QIP 
CIPP .. 

.,CF 

uCTK. CTNETK' 
D, DEM. V 

DreCKiM . 
DF . ; ; 

.Elim. 

ENV '^-^ 
'.EXP. ^ " ■ . 

F ^ 

FCN • 

F '-■ 'ratio: 
f' 1?est.- 

FTo , . : ■ - 
*. Fwrc 



GP- 
GSI 



• Air- to-Air . ■ . • 

" Aircraft , " _ . ' ' . , • ^ 

- Air Combat Engagement Sirmul^tor" 
Air Comba;t Maneuvering •'; ; 

- Air Combat Maneuvering- Ins trulnentar'tf^^^n 
Range ' . ' ■ ' . , ' . ' 

-'Air Forces Human Resources Laboratory - 

- Average ' . • . ' 

- ^Basic" flying maneuvers . , . 

- Chief Iris tructcir , Pilotf^^ '\ « . 

- Chi4f^ Ins tructot Pilot -predictions of/ 
turkey ' shoot ranking ' * 

^"^'^ . ' ■ ■ 

-/Classification Fun.ction • 

, - Cin^track-;^ exercise in tra.cking maneuvers- 

-. Dem'o graphic data .^ . 

^'^-•"Discriminant analysis- program ^ used ^ 

Degrees' of -freedofn . ' . . 

: - Eliminated (ors) f rom Turkey Shootv 

- Envelope 

Expand,ed C>List of variables) 

- Friday scoring data 

- Function . ' . ' , . . 

- Variance ^between' groups divided by 
variance/ v^i thin -groups ■ ' ■ , 

- Test of* significance used" in analysis 
o£ variance " *' ' 

' . . ^ " ^^ " . ' ^ ■ ^' 

- Flight Training Operations 

- Fighter-- We^^pons Instructor Course > 
Acceleration relative, to tliat --of \gravi ty 

- ^Greather than - : 

~ Group' . [ 

■ ■'/ ■ ■ ' 
Good Stick"Index * ' ^ 
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»1 

H-M];S 



3^ 



ViXM, HDVON , 

INT". txK^ 
IP ^ 

LBS ♦ ■ 



mIS'.err 

N 

0/D 



OTOPS 
PANG 

Pred. 

R 
r2 

R-MIS 
R.U. 

S 

SAAC 
SA^^ °, 
SR " 



- Hypothesis where P„ ^ P_ 

- Heat missile ^ x \ 

- Hypothesis where P^ = P^ 

- Head-on exercise 
Internal • ^ , 

T Instructor Pilot 

^ound9 (fuel) . 
-..Less than - 

iday' ^%?Dring data ' 

- Average pointing e;ifror in Mils 
I- Sarjiple size * , 

-^pJE^'tio of offensive, time (target in f roht ' 
.hemisphere of subject aircraft) to defen- 
sive time % s 

Opaque Target Optical Projector System. 

- ^?ointing Angle Advantage** (Time in envelope) 

- Proportion of correct classif icatibns . . 
using data withinTthe experiment 

-Prediction (s) -or(s) . , 

. ■ ■ , ^ ' ^ ^ . ^ . . • 

- Proportion of correct classifications 

using test data from outside the experi- 
ment, ' ' ' 



- Correlation coefficient 

- doeffioientj of detennination 

- Radar missile / 

- Runner ( s) -Up bf Turkey Shoot--^ 

- Standard (Jeviation 

- Simulator for Air-to-Air Combat 

- Surf ace-to-Air Missiles 

- Slant range 
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LIST OF abbreviations; ACRONYMS AND SYMBOLS . (Cont. ) . 



ss 

TAG 

TAG AGES I 
II 

'tas 

TFWC 

T.S., TS 

TTFK 



VAR 
2 

a 
W 

Win* 

X 

2 

X 
X. 



- Sums of squares \ 
. - Tafctical' Air Gommand 

- Simulator training^^^program at Vought 
<^t Luke Air Force^ Base ' . . . ' 
•Training' and simulation 

- Tactical Fighter Weappns Genter / 

- Turkey Shoot 

TP 

- TimiB from start of engagement to firsts 
kill ' - ■ . • 



-> Variable ' ^ ; 

- Vi^riance ^ 

- Wednesday scoring data 
Winner (sy of Turkey Shoo^: 

- Sample mean 

- Ghi-Square test statistic' 

- VariabJ|e quantities 

- Dependent variajale: quantity 
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